[caret-users] 64bit Linux and performance of Error Correction while Segmenting

[Donna Dierker]

Hi Dr. Ludwikow,

See inline replies below.

Donna

On 07/21/2009 04:59 AM, eep235 at bangor.ac.uk wrote:
> Hi,
>
> I am experiencing some long waiting times while running segmentation  
> on a pre-packaged Caret 5.61. Today it amounted to 1 h (when ran  
> caret5), the other day to 20 h (when ran caret5_mesa). 
One hour is not terribly long; 20 hours is way too long.  Was this 
exactly the same cropped input volume under both scenarios, with only 
the caret5 vs caret5_mesa difference?  If so, I'm very surprised, 
because I think the only difference is the GL libraries that get linked, 
and since segmentation is more of a crunch-the-volume thing than 
something display-intensive, I wouldn't expect much GL involvement.
> It is  
> especially the Auto Error Correction stage that takes long to compute.  
>   
Absolutely true.  And if you don't mind fixing handles, then turn it off.
> Apparently, the same dataset on a Mac takes approx. 20-30 min to be  
> processed.
>   
This also surprises me -- unless it is a brand new Mac with Xeon processors.
> The machine is x86_64 GNU/Linux Ubuntu running 2.6.28-14-generic  
> #46-Ubuntu SMP kernel. CPU is Intel Xeon X5355 (quad core, @2.66 GHz  
> each, ca. 5300 BogoMIPS each; model 15, stepping 7). 4 GB RAM, 12 GB  
> swap.
>
> top reveals that caret5 is grinding only one of the cores (@100%),  
> however, I am given to understand that this is only to be expected  
> ("User's Manual and Tutorial").
>   
Yeah, the segmentation process is not multi-threaded, and it would 
probably be hard to make it so.

The caret_stats stuff, however, does take advantage of multi-threading 
(permutation tests).
> As I was only introduced to Caret a few days ago, my first suspicion  
> is that I have been making some silly mistakes. Are there any obvious  
> and not-so-obvious things to check that would explain such long  
> computation time?
>   
Here are some suggestions:

* Adjust your expectations.  Try running Freesurfer on your cases and 
see how long that takes.  (Don't get me wrong; I know it's doing a lot 
for you.)  One hour isn't too bad if it's fixing most of your handles.

* Turn off automatic error correction if you can live without it for 
your purposes (or you don't mind fixing handles).

* Don't use caret5_mesa, if all other things were equal when you ran 
those two trials.

* Run multiple segmentations (in separate caret sessions)  at a time, to 
take advantage of multiple threads.  Note: I'd put the hemispheres in 
separate LH and RH subdirectories, to prevent intermediate files from 
getting trampled on.

* Get a Linux box with faster processors.  While 2.66 is not slow, I 
suspect the Mac processors might be faster.

* If a segmentation takes longer than a hour, then use a separate Caret 
session to view the initial segmentation overlaid on the anatomical 
volume.  If the skull or cerebellum is still attached, then this will 
create quite a feast for the handle fixer; it will take forever; and the 
end result will be pretty worthless.  At that point, you need to figure 
out how to get rid of the non-cortical junk.
>
> With kind regards,
> Marek Ludwikow
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
>
>