The Geek Parade

On today, the high holy day of, err, democracy

Tue, 04 Nov 2008 17:59:26 GMT

not me: I voted absentee. And you?

David Foster Wallace is dead at 46

Sun, 14 Sep 2008 05:54:05 GMT

Wallace apparently committed suicide, sez the Times. Man, he's basically my age, though, ah, spectacularly more productive. Infinite Jest isn't one of my favorite novels, but it's certainly one of the books that's shaped how I understand wit. Someone who wrote novels as dark as Wallace's could have had a pretty dark vision of his own.

Anyone for a quick game of Eschaton?

Resquiat in pace, David. I'll read your books again and with a new eye.

savagery

Tue, 26 Aug 2008 23:00:04 GMT

If your life feels like this, I understand. This is the refugee camp where stuffed mice are protected from the grasping razor-sharp kitteh claws by a few inches. But the kittehs are growing as fast as their stomachs can turn $4/lb cat food into kitteh poo. Growth is not good for refugee stuffed mice.

goatse, disney, and Rick.

Sun, 24 Aug 2008 22:15:19 GMT

I think it's very funny that in a Digg post showing Disney pix that imply the goatse horror is commented upon by a RickRolling. As a meta-commenter said, "goatse was the original, slightly more vicious Rick Roll." Each generation has to re-invent sex, drugs, and apparently goatse.

I think reality is slipping away

Tue, 19 Aug 2008 07:12:03 GMT

Using Photographs to Enhance Videos of a Static Scene from pro on Vimeo.

You take crappy video and a few high-res digital pix. This software mixes the photo detail & exposure into the video, creating scarily great video. Whoa.

pix of piglandia & kittehs

Tue, 05 Aug 2008 08:34:04 GMT

Here's where I spend most of my waking hours:

The battle for alfa-kitteh shakes the heavens, or at least annoys the downstairs neighbors:

They keep Stanyan Street under close patrol:

baby's got game

Sat, 02 Aug 2008 22:31:30 GMT

As humans, we have many reasons to be glad we're not field mice. Wunnie and Zirrz provide me with daily reminders.

ready for vogue?

Wed, 30 Jul 2008 07:45:31 GMT

haz to ware collar for 3 moar dayz.

mass murderer is a right-wing nut? really?

Wed, 30 Jul 2008 04:48:55 GMT

"Jim Adkisson, the 58 year old man being held in a Knoxville, Tennessee jail on murder charges stemming from a shooting during a children's musical at the Tennessee Valley Unitarian Universalist Church on Sunday, is said to have had a array of right-wing political books in his home, along with brass knuckles, empty shotgun shell boxes, and a handgun discovered by police who searched his home."

This we learn at Raw Story. I'm sure we'll hear howls of denial from Bill O'Reilly, Michael Savage, and Sean Hannity -- whose books were found, copiously annotated, in Adkisson's house -- about how their hate speech is unrelated to this killing. Um, no. Why did this pathetic, bitter, unemployed man turn his anger on liberals? Because that's what the Right does: blame liberals for the terrible but predictable outcomes of their harebrained and hateful political schemes.

Let me be clear -- assholes like these guys should have the right to say anything they want, however revolting it is. However, it seems to me that they should be morally compelled to take responsibility for what this man has done essentially in their name. An apology, perhaps? Maybe a bit of public reflection on the relationship of their statements to the killer's writings?

Imagine how the Right would react if a nutcase shot up a church, and the cops then found books by Eric Alterman, Barbara Ehrenreich, and Barack Obama in his house?

dsl and my apartment

Thu, 03 Jul 2008 22:02:01 GMT

I love Speakeasy DSL. They tell the truth, they don't mess around with your connection, they treat you like a valued customer -- quite the opposite of AT&T who has consistently treated me like a victim in their clutches. But it's going to be 2-3 weeks before the service is activated. Yick: how can I feel at home without connectivity?

Anyway, I can tether my Treo, as I've discussed before. It's slower than DSL, it drops occasionally, but it will do svn up and get my mail. I had a little trouble with OSX: Verizon told me that they don't support the Palm 700p and Mac. It's not that big a deal, though: MacNerd explains all.

I'm voting Republican

Mon, 30 Jun 2008 06:26:44 GMT

so that my little Caitlyn can be in a classroom with at least 30 other children. That way she can be challenged by fighting for attention.

because:
we need more minorities in prison.
I just don't feel that I deserve health insurance.
sometimes the Constitution is one big pain in the ass.

R performance: linux v mac

Sun, 29 Jun 2008 04:53:33 GMT

A newly-converted-to-mac friend of mine complained to me that an R script computing numeric solutions to 10k integrals is much slower on his new Mac than it was on Windows. Part of his problem, as he admits, is that the new Mac is a 1.6GHz CoreDuo MacAir while the old Dell is a 2.8GHz P4. It seems that the dual cores don't do that much good for R, even with Apple's highly optimized vecLib.

There's some good benchmarking on how the OSX kernel behaves relative to linux, and more good benchmarking about how R performs using different versions of the Basic Linear Algebra Stuff (BLAS -- ok I made up the last word).

There are various BLAS libraries, in addition to the standard one that ships with R and the Apple vecLib. The best of the lot is written by Dr Kazushige Goto at UT-Austin. The R folks make it easy to test different BLAS implementations (scroll down to section A.3.1.5).

This post is lame benchmarking. I ran the following snippet 3x in various combinations of operating systems, hardware, and BLAS libraries.


str <- rep("foo", 10000)
coco<-matrix(rnorm(1000000),1000,1000)

system.time(for(i in 1:1000) nchar(str))
system.time(for(i in 1:100) x <- rnorm(5e5))
system.time(x <- as.Date(rep("01-01-2005", 100000), format = "%m-%d-%Y")) 
system.time(coco2<-eigen(coco))

The combinations I used are below:


Mac OSX 10.5.2, 2.4 GHz CoreDuo, 4Gb 667 MHz SDRAM; 
     [a] R 2.6.2 (2008-02-08) with Goto blas
     [b] R 2.6.2 (2008-02-08) with Apple vecLib blas
     [e] R version 2.7.1 (2008-06-23)* with Goto blas
     [g] R version 2.7.1 (2008-06-23)* with standard blas
     [h] R version 2.7.1 (2008-06-23)* with vecLib blas
     [i] R version 2.7.1 (2008-06-23)+ with vecLib blas 
     [j] R version 2.7.1 (2008-06-23)+ with Goto blas 

Ubuntu Linux 2.6.22-14-generic SMP, 2.6 GHz Athlon 64, 
     [c] R version 2.5.1 (2007-06-27)+ [run 3x, min times below], standard blas
     [d] R version 2.7.1 (2008-06-23)* [run 3x, min times below], standard blas
     [e] R version 2.7.1 (2008-06-23)* [run 3x, min times below], Goto blas

* = my compile
+ = CRAN binary

And these are the results:


rep 1000              100                1000000           1
fun nchar(str)        rnorm(5e5)         as.Date(...       eigen(coco)
[a] 0.88  0.05  0.93 | 7.49 0.42  7.971 | 4.71  0.03 4.78 |  5.67  0.20  5.99 
[b] 0.88  0.05  0.95 | 7.49 0.39  7.936 | 4.81  0.02 4.90 |  6.74  0.82  6.56 
[f] 0.96  0.04  1.02 | 8.58 0.40  9.200 | 4.76  0.01 4.81 |  5.72  0.17  6.00 
[g] 0.95  0.05  1.05 | 8.59 0.43  9.204 | 4.76  0.01 4.81 | 12.60  0.19 13.05  
[h] 0.95  0.05  1.04 | 8.59 0.44  9.164 | 4.75  0.03 4.90 |  6.77  0.82  6.58  
[i] 0.93  0.04  0.97 | 7.48 0.40  8.036 | 4.70  0.03 4.77 |  6.81  0.83  6.43  
[j] 0.92  0.04  0.98 | 7.48 0.39  8.043 | 4.72  0.01 4.77 |  5.74  0.16  5.97
                                                                          
[c] 0.21  0.02  0.23 |11.94 0.30 12.261 | 0.22  0.08 0.30 | 16.98  0.14 17.19
[d] 1.69  0.02  1.72 |14.88 0.48 15.395 | 1.85  4.72 6.59 | 16.08  0.16 16.27 
[e] 1.81  0.02  1.84 |15.19 0.40 15.609 | 1.85  4.27 6.13 |  8.35  0.15  8.51

The table doesn't look very nice and I don't want to make it html. Anyway, the results are strange but apparently consistent: R version 2.7 is slightly slower than 2.5; Goto is faster than vecLib which is faster than R's blas; there are unknown tunings that make CRAN's binaries faster than stuff I compile by hand; and it seems that the Mac is faster than Linux in the mathematical transformations but Linux is faster in the string manipulations.

Take these results with a gigantic grain of salt: the machines are different, little compile tweaks could affect all sorts of things. There's an optimized LAPACK library also built into the Linux 2.5.1 version, but that doesn't explain the radically better nchar result. Could be linking against Doug Lea's malloc? Shouldn't be, since the nchar() call is so tiny it wouldn't be triggering a malloc on any platform.

So, the main conclusion is: use Goto's blas. Here's how I did it. Get his tarball from the page above, and unpack it in /usr/local/src; make && make install && cd exports && make dyn. The resulting file is libgoto_core2-r1.26.dylib: this is your new BLAS library.

Go over to /Library/Frameworks/R.framework/Versions/2.x/Resources/lib (where the x is 5, 6, or 7, depending on your version of R). Copy libgoto_core2-r1.26.dylib here, and make a symbolic link ln -s libgoto_core2-r1.26.dylib libRblas.dylib.

Note: if there is already a file called libRblas.dylib, you should move it to a new name so your symlink doesn't overwrite it. That's the native R blas, which is essentially useless. Changing to the Goto blas takes <5 minutes, and decreases matrix manipulation times by about 15%, but YMMV.

numerological reflections

Thu, 26 Jun 2008 17:52:12 GMT

with my recent birthday, my age is now evenly divisible only by itself and one. I suppose that means I'm in the prime of my life.

My new apartment

Mon, 23 Jun 2008 20:05:18 GMT

is in San Francisco, but nonetheless, this image is, to a first approximation, where I now live:

Jesus Trucking Christ

Wed, 04 Jun 2008 03:17:20 GMT

I'll tell ya, that Christ guy, he kicks some serious ass.

freedom isn't free

Sat, 24 May 2008 04:01:55 GMT

From a friend:

Show your support for the Supreme Court's decision to legalize same sex marriage by calling the Governor. Please pass this on to EVERYONE! Lots of people are calling in opposition- we need to make a huge showing of
support.

It couldn't be easier to vote: please take 15 seconds right now to do it! To vote in support of the Supreme Court's decision on LGBT marriage:

1. call 1-916-445-2841
2. press 1, 5, 1, 1

After you've done this send it on to all supporters you know.

Governor Arnold Schwarzenegger is receiving large numbers of phone calls from the supporters of "Limits on Marriage" (The constitutional amendment banning same sex marriage in California) about his comment to a group of Log Cabin Republicans that he wouldn't support a constitutional amendment. We don't want him to go back on his word. He must hear from us, too.

Call, email or write him and thank him for his declining to support this initiative. (they don't ask what state you are from)

To email go to to the governor's webmail site, hit e-mail tab and select Supreme Court decision on Same Sex Marriage. On the next screen you must click PRO. It's a little confusing since we're talking about the initiative for which we are con. Then just a couple of words of thanks are all you need. No long letter which no one will read anyway.

paper mail:
StateCapitolBuilding
Sacramento,CA95814

or direct:
Phone: 916-445-2841
Fax: 916-558-3160 ( new number )

There it is, folks. As Our Media taught us to say, "Just Do It."

religion in ferment

Fri, 16 May 2008 19:00:15 GMT

i can haz existence?

Ceiling cat iz in my eschatology, messin' wif my destinee.

ps. And now there's theology , and serious heretics.

Gay marriage ban overturned in CA!

Thu, 15 May 2008 20:05:45 GMT

Sometimes I'm just most especially extremely delighted to be a Californian, living in SF no less. From the Court's decision, rendered today:

We therefore conclude that in view of the substance and significance of the fundamental constitutional right to form a family relationship, the California Constitution properly must be interpreted to guarantee this basic civil right to all Californians, whether gay or heterosexual, and to same-sex couples as well as to opposite-sex couples.

To which any rational person simply has to agree (for select values of "rational," I suppose).

indeed.

Mon, 05 May 2008 15:33:26 GMT

The message here is that we all have our crazy friends, and shouldn't be judged by them. Yep, one of my crazy friends sent me this, and I couldn't have said it better myself.

too long

Mon, 05 May 2008 02:20:24 GMT

my bad. But in any case, I can spread a li'l lol-love:

long time no blog

Sat, 19 Apr 2008 01:48:46 GMT

I've been working with genetic algorithms in an attempt to optimize stratification of an arbitrary data space. I created 1000 random solutions (each a combination of randomly-generated strata), and then ranked them by a fitness measure. The 100 most fit solutions passed into the next generation. Then 900 new solutions were "bred" by mixing strata from two randomly-chosen solutions (more fit solutions had higher probabilities of being selected for pairing). I let it run for 11 hours which turned out to mean 82 generations (single-threaded, one core). The results are below.

(click to see a bigger image).

A spoiler: after this run, I abandoned this direct line of work. Nonetheless, a couple of conclusions.

1) The fitness measure is hosed (that's the technical term they teach you just before you get your PhD). It's calculated as pc_data*pc_cells/max_cells*n_strata*mn_data. However, the terms aren't scaled evenly, so the variable with the greatest absolute variance (n_strata) totally dominates the fitness measure. D'oh! Easy enough to fix, but, well, it takes 10 hours to do a run.

2) It's very interesting how sensitive the selection really is to fitness. The size of strata was initially distributed as 1-n cells, with the i-th selection being drawn from a zero-truncated normal distribution mu=2, sigma=3 (results were truncated to integers). The graph titled 'n_strata' shows the distribution of stratum counts among the best 100 solutions in each of 82 generations. n_strata has a lot of range -- 600-1400 -- so it drives the fitness measure to minimize stratum size. pc_data, the percent of all the data contained in strata with valid estimates, does not improve much, but that hides a cool fact:

3) max_cells is the maximum number of cells in a single stratum in a solution, and mn_data is the mean data elements per stratum. Note that these are necessarily very closely correlated with n_strata. There are likely interaction effects that are responsible for the very rapid move to small strata.

3) Nevertheless, even with a bad fitness measure, there's a clear positive trend in fitness over generations: the genetic selection process first drove to a very dispersed population of small strata without valid estimates. But very slowly, solutions with better pc_data values began to influence the fitness selections. The n_strata/mn_data/max_cells combinations were maxed, but within the constraints established in the first twenty generations, the algorithm started to find solutions that improved pc_data without increasing stratum size.

4) The genetic algorithm works like crazy. I tuned the fitness function in a dumb way, but the genetic algorithm did a much closer approximation to "what I really wanted" than any other algorithm I've used.

5) This said, it's unsurprising that the maximum fitness value turned up in the first generation and was never beaten. For the record, max(fitness)=3.22. If the mean fitness continued to increase at the rate set in generations 11-82, the mean fitness would exceed the initial optimum after approximately 1400 more generations. It's hard to imagine that pc_data could continue to increase much farther without triggering limiting reactions in the n_strata components. Interesting, but not immediately promising.

6) Sure, I could play around with a model in which a database server hands out solutions to worker threads, and I could then use multiple OSX and linux machines (or an Amazon EC2 cloud) to run threads, but we're still talking like 14 core-hours per 100 generations. I could probably access about 10 cores relatively easily, but still, it's not like having an answer in a fraction of a second. I played around with a few models like this for a couple of weeks, but it ends up needing a lot of infrastructure for an experiment.

7) I think that this analysis implies that varying fitness functions will generate varying distributions of strata. This could be very interesting.

This is worth coming back to, but not immediately. There are more promising directions ...

continuing education

Wed, 20 Feb 2008 17:15:48 GMT

As a trained IT professional, I consider it essential to keep up on my skillz. That's why I watch instructional videos on the interwebs, like this one.

In this episode, Donnie teaches us how to remove the wedding band from the woman's finger, because, as he points out, "it has lost all meaning and value." I definitely do not recommend episode #2 where he puts the kitten (aka "piss_factory.jpg") in the trash bag just because his wife brought the kitten home without asking.

Tue, 19 Feb 2008 17:00:32 GMT

keeping track of myself in Leopard

Tue, 19 Feb 2008 16:53:44 GMT

I find that I get a lot more done if I keep a log of what I've already done: seeing the list of done stuff growing is satisfying. This is timely (heh heh heh), because it turns out that due to a new contract at my company, I have to keep a daily timesheet of my hours (++ungood).

Many applications later, I chose Memoire for the log tool. It keeps the entries organized by day in a calendar view, but using the search field, I can also look at the notes in a list view. If necessary, I can export into clean text for some kind of post-processing.

The only weak link is the Spotlight integration: while Spotlight will find some note, Spotlight is file-focused. That means Spotlight doesn't know to jump to a particular entry. Surely there's an Automator action or Quicksilver action that could tie Spotlight to Memoire's search function ...

what's a blog without banner ads?

Mon, 18 Feb 2008 19:03:11 GMT