Analysis of benchmarks

As soon as the benchmarks have been submitted you can request a summary of the current performance. If a job has not yet finished, not yet started or crashed, MDBenchmark notifies you and marks the affected benchmarks accordingly.

Retrieving the results

The benchmark results can be retrieved immediately after they have been submitted, even if the jobs have not yet started. To do this, simply run:

mdbenchmark analyze

This will do two things for you:

  1. Print the current performance results for all benchmarks found recursively in the current directory.
  2. Save the above performance results to a .csv file.

The printed results look like this:

                     module  nodes  ns/day  run time [min]    gpu        host  ncores
0  gromacs/2016.4-plumed2.3      1  10.878              15  False       draco      32
1  gromacs/2016.4-plumed2.3      2   21.38              15  False       draco      64
2  gromacs/2016.4-plumed2.3      3  34.033              15  False       draco      96
3  gromacs/2016.4-plumed2.3      4       ?              15  False       draco       ?
4  gromacs/2016.4-plumed2.3      5   51.71              15  False       draco     160

The results above showcases that MDBenchmark displays jobs that have not finished, started or crashed with a question mark (?).

Defining a name for the CSV file

You can define the name of the output CSV file with the -o or --output-name option:

mdbenchmark analyze --output-name my_benchmark_results.csv

Narrow down results to a specific benchmark

Similar to the submission of benchmarks, you can use the --directory option to narrow down the performance analysis to a specific path of benchmarks or a single benchmark:

mdbenchmark analyze --directory draco_gromacs/2018.2

Plotting of benchmark results

MDBenchmark provides a quick and simple way to plot the results of the benchmarks, giving you a .pdf file as output. To generate a plot simply use the --plot option:

mdbenchmark analyze --plot

Warning

The plotting function currently only allows to plot CPU and GPU benchmark from the same module, and also assumes that benchmarks were always performed with CPUs. This behavior will be fixed in a future release. If you want to compare different modules with each other, either use the --directory option to generate separate plots or create your own plot from the provided CSV file.

Plot the number of cores

You can customize the top of your plot with the --ncores option. It accepts an integer value, referring to the number of cores per node. If the option is not given, MDBenchmark will try to read this information from the log file.