MLBench Benchmark Implementations

MLBench contains several benchmark tasks and implementations. Tasks combinations of datasets and target metrics, whereas the implementations are concrete models and code that solve a task.

For an overview of MLBench tasks, please refer to the Benchmarking Tasks Section

Closed Division Benchmark Implementations

A Benchmark Implementation is a model with fixed hyperparameters that solves a Benchmark Task.

Image Recognition

1a. ResNet, CIFAR-10

PyTorch Cifar-10 ResNet-20 Open-MPI
Framework:PyTorch
Communication Backend:
 Open MPI (PyTorch torch.distributed)
Distribution Algorithm:
 All-Reduce
Model:ResNet-20
Dataset:CIFAR-10
GPU:Yes
Seed:42
Image Location:/pytorch/imagerecognition/openmpi-cifar10-resnet20-all-reduce/
PyTorch Cifar-10 ResNet-20 Open-MPI Scaling

Resnet 20 implementation with scaling Learning Rate

Framework:PyTorch
Communication Backend:
 Open MPI (PyTorch torch.distributed)
Distribution Algorithm:
 All-Reduce
Model:ResNet-20
Dataset:CIFAR-10
GPU:Yes
Seed:42
Image Location:/pytorch/imagerecognition/openmpi-cifar10-resnet20-all-reduce/
PyTorch PASCAL Challenge 2008 Logistic Regression Open-MPI

Logistic Regression implementation

Framework:PyTorch
Communication Backend:
 Open MPI (PyTorch torch.distributed)
Distribution Algorithm:
 All-Reduce
Model:Logistic Regression
Dataset:PASCAL Challenge 2008 epsilon
GPU:Yes
Seed:42
Image Location:/pytorch/linearmodels/openmpi-epsilon-logistic-regression-all-reduce/
Tensorflow Cifar-10 ResNet-20 Open-MPI
Framework:TensorFlow
Communication Backend:
 Open MPI
Distribution Algorithm:
 All-Reduce
Model:ResNet-20
Dataset:CIFAR-10
GPU:Yes
Seed:42
Image Location:/tensorflow/imagerecognition/openmpi-cifar10-resnet20-all-reduce/