fit_generator() to save model with least validation loss

Question

How do I use keras function fit_generator() to train and simultaneously save the model weights with lowest validation loss?

Answer 1

You can set save_best_only=True while defining checkpoint:

from keras.callbacks import EarlyStopping, ModelCheckpoint
  
early_stop = EarlyStopping(
     monitor='loss',
     min_delta=0.001,
     patience=3,
     mode='min',
     verbose=1
)
checkpoint = ModelCheckpoint(
     'model_best_weights.h5', 
     monitor='loss', 
     verbose=1, 
     save_best_only=True, 
     mode='min', 
     period=1
)

Now when fitting the model just include the parameter callbacks = [early_stop,checkpoint]. It will save the weights with the lowest validation loss.

model.fit_generator(X_train, Y_train, validation_data=(X_val, Y_val), 
      callbacks = [early_stop,checkpoint])

Saving Model Architecture

If you want to save your model architecture too, you will need to serialize the model to JSON:

model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)

Finally load the model with architecture and weights:

# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model_best_weights.h5")
print("Loaded model from disk")
 
# evaluate loaded model on test data
loaded_model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
score = loaded_model.evaluate(X, Y, verbose=0)

Please refer to: https://machinelearningmastery.com/save-load-keras-deep-learning-models/

Answer 2

You can save the model weights using the following code.

model.save_weights('weights.h5')

And you can save the architecture of the model using the following code:

model.save('architecure.h5')

If space is not a problem then you can store all the models and select the one which has the lowest validation loss.

Or you can use callbacks after each epoch to evaluate validation loss and take the model which has the current lowest loss on validation data. This can be done by referring to the following link. In this example just change the data passed to TestCallback and have a variable to store the current minimum validation loss.

class TestCallback(Callback):
def __init__(self, test_data):
    self.test_data = test_data

def on_epoch_end(self, epoch, logs={}):
    x, y = self.test_data
    loss, acc = self.model.evaluate(x, y, verbose=0)
    print('\nTesting loss: {}, acc: {}\n'.format(loss, acc))

model.fit(X_train, Y_train, validation_data=(X_val, Y_val), 
      callbacks=[TestCallback((X_test, Y_test))])

Using callbacks example