?

Log in

No account? Create an account
 

A happy ending to my last rant: I did some hacking around (with… - F*cking with Clusters

About A happy ending to my last rant: I did some hacking around (with…

Previous Entry Mar. 3rd, 2026 @ 10:58 pm Next Entry
A happy ending to my last rant:

I did some hacking around (with suggestions from the inimitable pabw), and started my kernel computation again. Last time, it ran for 9 days and failed when the power died. The ETA was around 38 days. This time.... 43 hours. Who now has the complete kernel matrix for 17 gigs of fMRI data? *Incrementally cached*? That'd be me. Bow, all ye data-matrix less peons.
Current Mood: jubilantjubilant
take a penny
[User Picture Icon]
From:maerdi
Date:March 4th, 2006 06:58 am (UTC)
(Link)
By the way, what are you doing with all that fMRI data? (A link to a past post would be more than sufficient if you've already explained it all before.)
[User Picture Icon]
From:avani
Date:March 6th, 2006 02:52 pm (UTC)
(Link)
Well, we have to use this dataset because the guys who are interested in it are giving us a ton of cash, but the purpose is to test different types of classifiers against each other (in this case, un-scaled support vector machines against an in-house dimensionality reducer under a Bayes net)
[User Picture Icon]
From:pabw
Date:March 4th, 2006 09:27 am (UTC)
(Link)
Hm.
0.5 * 40 * 17 * 1024 * 1024 * 1024 / (43 * 3600) = 2358347.6754521965

~ 2.25 MB/s. It should be possible to do better, even in python. I'd expect 30-50 MB/s from the hard drive...

[User Picture Icon]
From:pabw
Date:March 4th, 2006 10:23 am (UTC)
(Link)
Or maybe that should be 17*1024*1024*1024 * 40*21 /(43*3600*40) = 2476265
or ~2.36 MB/s.
[User Picture Icon]
From:amoken
Date:March 4th, 2006 06:53 pm (UTC)
(Link)
Wahoo!! Shiny. :D
(take a penny)
Top of Page Powered by LiveJournal.com