How to fix LSTM Layer Error - Image Classification

Question

Currently I'm working on an image classification issue and created the following code based on a tutorial online - Image Classification using Keras.

The code works fine, but when addind the LSTM layer it has problems with the input_shape and I'm unable to figure out the solution:

ValueError: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=4

The code:

img_width, img_height = 224, 135
train_dir = './train'
test_dir = './test'
train_samples = 46822
test_samples = 8994
epochs = 25
batch_size = 16

input_shape = (img_width, img_height, 3)

model = Sequential() 
model.add(Conv2D(32, (3, 3), input_shape = input_shape, activation = 'relu'))
model.add(LSTM(3, return_sequences=True, input_shape = input_shape))
model.add(AveragePooling2D(pool_size = (2, 2)))
model.add(Flatten())
model.add(Dense(units = 128, activation = 'softmax'))

model.compile(loss ='categorical_crossentropy', 
              optimizer ='adam', 
              metrics =['accuracy'])

train_datagen = ImageDataGenerator( 
            rescale = 1. / 255, 
            shear_range = 0.2, 
            zoom_range = 0.2, 
        horizontal_flip = True) 

test_datagen = ImageDataGenerator(rescale = 1. / 255)

train_generator = train_datagen.flow_from_directory(train_dir, target_size =(img_width, img_height), batch_size = batch_size, class_mode ='categorical')

validation_generator = test_datagen.flow_from_directory(test_dir, target_size =(img_width, img_height), batch_size = batch_size, class_mode ='categorical') 

model.fit_generator(train_generator, 
    steps_per_epoch = train_samples // batch_size, 
    epochs = epochs, validation_data = validation_generator, 
    validation_steps = test_samples // batch_size) 

Extra info:

The size of input_shape = (224,135,3)

In the train and test folder are each 3 sub folders that contains a set of images based on Human Movements sequences.

The mentioned error does provide some Google results, but those didn't provide a solution in my case --> I've tried changing the input_shape for the LSTM layer to various options like (224,3) or any variant, etc.

I'm probably overseeing one stupid thing and hope someone here might have an idea?

Answer 1

Add TimeDistributed(Flatten()) layer before the LSTM layer.