[maker-devel] Hyperthreading

[Martin MOKREJŠ]

System Admin wrote:
> We are using maker in a cluster with mpich.  Currently hyperthreading is on and we use 'mpiexec -n <cores>' to start maker.  Our machinelist file for mpich specifies the total emulated cores for each node.
> With hyperthreading on, we have up to 256 total emulated cores available.
> 
> Which is the optimal scenario?
> 1. Use '-n 256'
> 2. Use '-n 128' with hyperthreading still on
> 3. Use '-n 128' with hyperthreading turned off

Go for 3. but make sure to disable *hyperthreading* in the kernel of the machines as well. I also disable multicore scheduler (which should again be helping if there are more long-term running processes than physical cores available and if some should probably share a cache). We do not have such jobs, hmmer and blast are mostly accessing data from memory, so the CPU cache is not much relevant for these.
  Hyperthreading only helps if jobs are lousy, waiting for some input/output etc., and in that case *it helps* if another process can be executed on the CPU core (hopefully not having same bottleneck). This is generally a helped in bad situations. You are after good setup, so disable hyperthreading in kernel, load only that many jobs equal to the number of physical CPI cores, and monitor performance. If jobs are starving, resolve the issue.

Martin