How to specify augmentation in TensorFlow Lite Model Maker?

Question

I'm trying to make an image classifier with the help of TensorFlow Lite Model Maker following this tutorial: https://www.tensorflow.org/lite/tutorials/model_maker_image_classification .

Everything works fine when I use the default values.

What I'm trying to do is to configure the hyperparameters, more exactly the augmentation hyperparamter: use_augmentation. https://www.tensorflow.org/lite/tutorials/model_maker_image_classification#change_the_training_hyperparameters

The documentation says:

use_augmentation: Boolean, use data augmentation for preprocessing. False by default.

Now my question is what data augmentation is used? And it is possible to configure this augmentations? Like I would like to use horizontal_flip, specify rotation and zoom range, etc.

Thanks in advance!

Answer 1

By going to the source code of tflite_model_maker.img and following the use_augmentation argument, I came here, where the augmentation is done. If the model is training, the augmentations seem to be random crop and flip. When evaluating, obviously no random crop or flips are done, but only center crop.

Answer 2

When you specify the model type, you can define the predefined augmentation policies like this:

spec = tflite_model_maker.model_spec.get("efficientdet_lite0")
spec.config.autoaugment_policy = 'v0'

The available policies are: ['v0', 'v1', 'v2', 'v3']. By looking at autoaugment.py, you can see the list if augmentations defined for v0

def policy_v0():
  """Autoaugment policy that was used in AutoAugment Detection Paper."""
  # Each tuple is an augmentation operation of the form
  # (operation, probability, magnitude). Each element in policy is a
  # sub-policy that will be applied sequentially on the image.
  policy = [
      [('TranslateX_BBox', 0.6, 4), ('Equalize', 0.8, 10)],
      [('TranslateY_Only_BBoxes', 0.2, 2), ('Cutout', 0.8, 8)],
      [('Sharpness', 0.0, 8), ('ShearX_BBox', 0.4, 0)],
      [('ShearY_BBox', 1.0, 2), ('TranslateY_Only_BBoxes', 0.6, 6)],
      [('Rotate_BBox', 0.6, 10), ('Color', 1.0, 6)],
  ]
  return policy