mlbench_core.evaluation

pytorch

criterion

Customized Loss Functions.

BCELossRegularized

class mlbench_core.evaluation.pytorch.criterion.BCELossRegularized(weight=None, size_average=None, reduce=None, l1=0.0, l2=0.0, model=None, reduction='mean')[source]

Binary Cross Entropy (BCE) with l1/l2 regularization.

Parameters

  • weight (Tensor, optional) – a manual rescaling weight given to each class. If given, it has to be a Tensor of size C. Otherwise, it is treated as if having all ones.

  • size_average (bool, optional) – Deprecated (see reduction).

  • default, (By) – the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average is set to False, the losses are instead summed for each minibatch. Ignored when reduce is False. Default: True

  • reduce (bool, optional) – Deprecated (see reduction). By

  • the (default,) – losses are averaged or summed over observations for each minibatch depending on size_average. When reduce is False, returns a loss per batch element instead and ignores size_average. Default: True

  • l1 (float, optional) – The scale of the L1 regularization. Default:

  • 0.0 –

  • l2 (float, optional) – The scale of the L2 regularization. Default:

  • 0.0 –

  • model (torch.nn.Module) – a pytorch model to be trained and

  • validated. –

  • reduction (string, optional) – Specifies the reduction to apply to

  • output (the) – ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied, ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: ‘mean’

MSELossRegularized

class mlbench_core.evaluation.pytorch.criterion.MSELossRegularized(weight=None, size_average=None, reduce=None, l1=0.0, l2=0.0, model=None, reduction='mean')[source]

Mean Squared Error (MSE) with l1/l2 regularization.

Parameters

  • weight (Tensor, optional) – a manual rescaling weight given to each class. If given, it has to be a Tensor of size C. Otherwise, it is treated as if having all ones.

  • size_average (bool, optional) – Deprecated (see reduction).

  • default, (By) – the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average is set to False, the losses are instead summed for each minibatch. Ignored when reduce is False. Default: True

  • reduce (bool, optional) – Deprecated (see reduction). By

  • the (default,) – losses are averaged or summed over observations for each minibatch depending on size_average. When reduce is False, returns a loss per batch element instead and ignores size_average. Default: True

  • l1 (float, optional) – The scale of the L1 regularization. Default:

  • 0.0 –

  • l2 (float, optional) – The scale of the L2 regularization. Default:

  • 0.0 –

  • model (torch.nn.Module) – a pytorch model to be trained and

  • validated. –

  • reduction (string, optional) – Specifies the reduction to apply to

  • output (the) – ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied, ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: ‘mean’

class mlbench_core.evaluation.pytorch.criterion.LabelSmoothing(padding_idx, smoothing=0.0, fast_xentropy=False)[source]

NLL loss with label smoothing.

Parameters

  • padding_idx (int) – Code for padding char

  • smoothing (float) – Smoothing value

  • fast_xentropy (bool) – Use apex.contrib.xentropy.SoftmaxCrossEntropyLoss

metrics

Utilities for measuring the performance of a model.

TopKAccuracy

class mlbench_core.evaluation.pytorch.metrics.TopKAccuracy(topk=1)[source]

Top K accuracy of an output.

Counts a prediction as correct if the target value is in the top k predictions, false otherwise, and returns the number of correct instances relative to total instances (0.0 to 100.0).

Parameters

topk (int, optional) – The number of top predictions to consider. Default: 1

__call__(self, output, target)[source]

Computes the precision@k for the specified values of k

Parameters

  • output (torch.Tensor) – Predictions of a model

  • target (torch.Tensor) – Target labels

Example

>>> m = nn.Softmax()
>>> input = torch.randn(10, 50)
>>> preds = m(input)
>>> targets = torch.randint(0, 1, (10,50))
>>> topk = TopKAccuracy(5)
>>> precision = topk(preds, targets)
Returns

float

property name(self)[source]

str: Name of this metric.

tensorflow

criterion

Define loss functions.

softmax_cross_entropy_with_logits_v2_l2_regularized

mlbench_core.evaluation.tensorflow.criterion.softmax_cross_entropy_with_logits_v2_l2_regularized(logits, labels, l2, loss_filter_fn)[source]

Return an op for computing cross entropy with weight decay.

The labels are assumed to be one-hot encoded. The loss filter function excludes some tensors from computing weight decay.

Parameters

  • logits (tf.Tensor) – input logits tensor.

  • labels (tf.Tensor) – input one-hot encoded tensor.

  • l2 (float) – size of weight decay

  • loss_filter_fn (callable) – filter function.

Returns

a scalar tensor

Return type

tf.Tensor

metrics

Define tensorflow metrics.

topk_accuracy

mlbench_core.evaluation.tensorflow.metrics.topk_accuracy_with_logits(logits, labels, k=1)[source]

Compute the top-k accuracy of logits.

Parameters

  • logits (tf.Tensor) – input tensor

  • labels (tf.Tensor) – input one-hot encoded tensor.

  • k (int, optional) – Defaults to 1. top k accuracy.

Returns

a scalar tensor of the accuracy (between 0 and 1).

Return type

tf.Tensor