How to configure the Keras Optimizer and Learning rate using config.yaml file?

Question

I have defined few parameters in my config.yaml like as below.

params:
  epochs: 10
  batch_size: 128
  num_classes: 10
  loss_function: sparse_categorical_crossentropy
  metrics: accuracy
  optimizer: SGD
  validation_datasize: 5000
  learning_rate : '1e-3

Now I am calling the same in my main. py as below.

config = read_config(config_path)
#  Create the model
LOSS_FUNCTION = config["params"]["loss_function"]
OPTIMIZER = config["params"]["optimizer"]
LEARNING_RATE = config["params"]["learning_rate"]
METRICS = config["params"]["metrics"]
model = create_model(LOSS_FUNCTION, OPTIMIZER, METRICS,LEARNING_RATE)

in my create model function if I user below way the code is failing.

def create_model(LOSS_FUNCTION, OPTIMIZER, METRICS,LEARNING_RATE):
    LAYERS = [
            tf.keras.layers.Flatten(input_shape=[28,28], name="inputlayer"),
            tf.keras.layers.Dense(300, name="hiddenlayer1"),
            tf.keras.layers.LeakyReLU(), ## alternative way
            tf.keras.layers.Dense(100, name="hiddenlayer2"),
            tf.keras.layers.LeakyReLU(),
            tf.keras.layers.Dense(10,activation="softmax", name="outputlayer")
    ]
    INPUT_OPTIMIZER = tf.keras.optimizers.OPTIMIZER(learning_rate=LEARNING_RATE)
    model_clf =  tf.keras.models.Sequential(LAYERS)
    model_clf.summary()
model_clf.compile(loss=LOSS_FUNCTION,
            optimizer=INPUT_OPTIMIZER,
            metrics=[METRICS])

' So have to one more time manually define and substitute.'

INPUT_OPTIMIZER = tf.keras.optimizers.SGD(learning_rate=1e-3)
    model_clf =  tf.keras.models.Sequential(LAYERS)

    model_clf.summary()
    model_clf.compile(loss=LOSS_FUNCTION,
                optimizer=INPUT_OPTIMIZER,
                metrics=[METRICS])

How to configure to take the config.yaml define optimzer value?. Thanks

Answer 1

I think you can do this with optimizers.deserialize You have to follow this format.

{'class_name': 'RMSprop',
 'config': {'name': 'RMSprop',
  'learning_rate': 0.001,
  'decay': 0.0,
  'rho': 0.9,
  'momentum': 0.0,
  'epsilon': 1e-07,
  'centered': False}}

I got this as output from optimizers.serialize

Note that optimizers.serialize outputs JSON, but you can still use YAML

As a use case example, you can create an object for your optimizer in your configuration file like this one

optimizer: 
  class_name: RMSprop
  config:
    learning_rate: 0.001

Then use it in your code like this

model_train.compile(optimizer=optimizers.deserialize(config['optimizer']))