'Custom metric for Face Alignment model, using Tensorflow 2.5

I want to add a Normalized Mean Error (NME) metric to model using TF. The NME is commonly used in the evaluation of the face alignment tasks. For that, I use the keras.metrics.Metric like below:

class NormalizedMeanError(tf.keras.metrics.Metric):
"""
Computes the mean error normalized by inter-ocular outer eye distance.
This is weighted by `sample_weight`.
If `sample_weight` is `None`, weights default to 1.
Use `sample_weight` of 0 to mask values.

Args:
    name: (Optional) string name of the metric instance.
    dtype: (Optional) data type of the metric result.
    den: (Optional) normalization type. If 'den' is None,
        we use the inter-ocular outer eye distance. Else if
        'den' is int, we use the setting value.

    The normalization value can be the root of the image area in pixels.
    In this case, zero division problems may not be encountered, unlike
    if we use the inter-ocular distance in case where both eyes are projected
    to close points in the image plane.
"""
def __init__(self,
             name='normalized_mean_error',
             dtype=None,
             den=None):
    super(NormalizedMeanError, self).__init__(name=name, dtype=dtype)
    self.normalization_value = den
    self.normalized_mean_error = self.add_weight(name='nme', initializer='zeros')

def update_state(self, y_true, y_pred, sample_weight=None):
    """
    Accumulates metric statistics.

    Args:
      y_true: The ground truth values.
      y_pred: The predicted values.
      sample_weight: Optional weighting of each example. Defaults to 1. Can be a
        `Tensor` whose rank is either 0, or the same rank as `y_true`, and must
        be broadcastable to `y_true`.
    """
    try:
        y_true = tf.cast(y_true, tf.float32)
        y_pred = tf.cast(y_pred, tf.float32)

        y_true = tf.reshape(y_true, [-1, 2])
        y_pred = tf.reshape(y_pred, [-1, 2])

        if self.normalization_value is None:
            # Outer eye landmarks
            left_outer_landmark = y_true[45, :]
            right_outer_landmark = y_true[36, :]

            # Inter-ocular outer eye distance
            self.normalization_value = tf.norm(left_outer_landmark - right_outer_landmark, ord='euclidean')

        l2_error = tf.norm(y_true - y_pred, axis=1, ord='euclidean')
        self.normalized_mean_error = tf.math.reduce_mean(l2_error / self.normalization_value)

    except Exception as e:
        print(e)

def result(self):
    return self.normalized_mean_error

I have some questions about it. First, when I use the model.evaluate(test_set) function, is the NME averaged over all y_true, y_pred of a batch? And at the end of this evaluation, is the NME is averaged over all the batch from the test set? So, is the normalized_mean_error on the screenshot the global average of my model performance over the test set? enter image description here



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source