mlbench_core.evaluation¶
pytorch¶
criterion¶
Customized Loss Functions.
BCELossRegularized¶

class
mlbench_core.evaluation.pytorch.criterion.
BCELossRegularized
(weight=None, size_average=None, reduce=None, l1=0.0, l2=0.0, model=None, reduction='mean')[source]¶ Binary Cross Entropy (BCE) with l1/l2 regularization.
 Parameters
weight (Tensor, optional) – a manual rescaling weight given to each class. If given, it has to be a Tensor of size C. Otherwise, it is treated as if having all ones.
size_average (bool, optional) – Deprecated (see
reduction
).default, (By) – the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field
size_average
is set toFalse
, the losses are instead summed for each minibatch. Ignored when reduce isFalse
. Default:True
reduce (bool, optional) – Deprecated (see
reduction
). Bythe (default,) – losses are averaged or summed over observations for each minibatch depending on
size_average
. Whenreduce
isFalse
, returns a loss per batch element instead and ignoressize_average
. Default:True
l1 (float, optional) – The scale of the L1 regularization. Default:
0.0 –
l2 (float, optional) – The scale of the L2 regularization. Default:
0.0 –
model (
torch.nn.Module
) – a pytorch model to be trained andvalidated. –
reduction (string, optional) – Specifies the reduction to apply to
output (the) – ‘none’  ‘mean’  ‘sum’. ‘none’: no reduction will be applied, ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed. Note:
size_average
andreduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will overridereduction
. Default: ‘mean’
MSELossRegularized¶

class
mlbench_core.evaluation.pytorch.criterion.
MSELossRegularized
(weight=None, size_average=None, reduce=None, l1=0.0, l2=0.0, model=None, reduction='mean')[source]¶ Mean Squared Error (MSE) with l1/l2 regularization.
 Parameters
weight (Tensor, optional) – a manual rescaling weight given to each class. If given, it has to be a Tensor of size C. Otherwise, it is treated as if having all ones.
size_average (bool, optional) – Deprecated (see
reduction
).default, (By) – the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field
size_average
is set toFalse
, the losses are instead summed for each minibatch. Ignored when reduce isFalse
. Default:True
reduce (bool, optional) – Deprecated (see
reduction
). Bythe (default,) – losses are averaged or summed over observations for each minibatch depending on
size_average
. Whenreduce
isFalse
, returns a loss per batch element instead and ignoressize_average
. Default:True
l1 (float, optional) – The scale of the L1 regularization. Default:
0.0 –
l2 (float, optional) – The scale of the L2 regularization. Default:
0.0 –
model (
torch.nn.Module
) – a pytorch model to be trained andvalidated. –
reduction (string, optional) – Specifies the reduction to apply to
output (the) – ‘none’  ‘mean’  ‘sum’. ‘none’: no reduction will be applied, ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed. Note:
size_average
andreduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will overridereduction
. Default: ‘mean’

class
mlbench_core.evaluation.pytorch.criterion.
LabelSmoothing
(padding_idx, smoothing=0.0, fast_xentropy=False)[source]¶ NLL loss with label smoothing.
 Parameters
padding_idx (int) – Code for padding char
smoothing (float) – Smoothing value
fast_xentropy (bool) – Use apex.contrib.xentropy.SoftmaxCrossEntropyLoss
metrics¶
Utilities for measuring the performance of a model.
TopKAccuracy¶

class
mlbench_core.evaluation.pytorch.metrics.
TopKAccuracy
(topk=1)[source]¶ Top K accuracy of an output.
Counts a prediction as correct if the target value is in the top
k
predictions, false otherwise, and returns the number of correct instances relative to total instances (0.0 to 100.0). Parameters
topk (int, optional) – The number of top predictions to consider. Default:
1

__call__
(self, output, target)[source]¶ Computes the precision@k for the specified values of k
 Parameters
output (
torch.Tensor
) – Predictions of a modeltarget (
torch.Tensor
) – Target labels
Example
>>> m = nn.Softmax() >>> input = torch.randn(10, 50) >>> preds = m(input) >>> targets = torch.randint(0, 1, (10,50)) >>> topk = TopKAccuracy(5) >>> precision = topk(preds, targets)
 Returns
float
tensorflow¶
criterion¶
Define loss functions.
softmax_cross_entropy_with_logits_v2_l2_regularized¶

mlbench_core.evaluation.tensorflow.criterion.
softmax_cross_entropy_with_logits_v2_l2_regularized
(logits, labels, l2, loss_filter_fn)[source]¶ Return an op for computing cross entropy with weight decay.
The labels are assumed to be onehot encoded. The loss filter function excludes some tensors from computing weight decay.
 Parameters
logits (
tf.Tensor
) – input logits tensor.labels (
tf.Tensor
) – input onehot encoded tensor.l2 (
float
) – size of weight decayloss_filter_fn (
callable
) – filter function.
 Returns
a scalar tensor
 Return type
tf.Tensor
metrics¶
Define tensorflow metrics.
topk_accuracy¶

mlbench_core.evaluation.tensorflow.metrics.
topk_accuracy_with_logits
(logits, labels, k=1)[source]¶ Compute the topk accuracy of logits.
 Parameters
logits (
tf.Tensor
) – input tensorlabels (
tf.Tensor
) – input onehot encoded tensor.k (
int
, optional) – Defaults to 1. top k accuracy.
 Returns
a scalar tensor of the accuracy (between 0 and 1).
 Return type
tf.Tensor