Input shape for Keras conv1D with sequential data

Question

I want to use Conv1D for making dilated convolution network to a sequential dataset.
So I tried Conv1D with Boston dataset.

from tensorflow.python.keras.layers import Conv1D, MaxPooling2D
from tensorflow.python.keras.layers import Activation, Dropout, Flatten, Input, Dense
from tensorflow.python.keras.models import Model
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split

boston = load_boston()
df = pd.DataFrame(boston.data,columns=boston.feature_names)
df['target']= boston.target
y = df['target']
X = df.drop(columns=['target'])
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=2)

def NN_model(data= X_train):
  #data = np.expand_dims(data, axis=2)
  inputs = Input(((data.shape)))
  x = Conv1D(2,2,dilation_rate=2,padding="same", activation="relu")(inputs)
  x = Flatten()(x)
  x = Dense(2048, activation="relu")(x)
  predictions = Dense(1)(x)
  model = Model(inputs=inputs, outputs=predictions)
  model.compile(optimizer="adam",loss='mean_squared_error')
  model.summary()
  return model
nn_model = NN_model(X_train)

The model had no errors, but when I tried

nn_model.fit(X_train,y_train)

and it rised an error as

ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (404, 13)

And I added this new line in function NN_model as

def NN_model(data= X_train):
  data = np.expand_dims(data, axis=2)

or

def NN_model(data= X_train):
  data = np.reshape(X_train.values, (-1, data[1], 1))

but it raised an error in the model as

ValueError: Input 0 of layer conv1d_1 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 404, 13, 1]

What should I do?

Answer 1

The problem is in how you are defining the input shape. From the keras documentation:

shape: A shape tuple (integer), not including the batch size

What this means is that the shape in your input layer should define the shape of a single piece of data, rather than the entire training dataset. inputs = Input(((data.shape))) is giving you the entire dataset size, in this case (404,13). However, each of your samples is really just a 13 length vector, so the shape you want is (13,1).

To get the desired input shape, you can use something like inputs = Input(((data.shape[1],1))). This ignores the first dimension (which gives you the number of samples).

The fix you tried in reshaping your data is actually a step you'll need to take before training. You need the input to model.fit to have shape (number of samples, 13, 1), so you'll need to reshape your data to match that, just like you did in your example:

X_train = np.expand_dims(X_train, axis=2)
nn_model.fit(X_train,y_train)

For future reference, the second error you got:

ValueError: Input 0 of layer conv1d_1 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 404, 13, 1]

is telling you that you your input is the wrong size to work with a Conv1D layer. This is because the Conv1D layer requires a 3 dimensional input. Assuming you are using a channels last format (versus channels first), this would be (batch, steps, channels). I.e. the first dimension is the batch size, the second dimension is the number of time steps (in this case 13), and the third dimension is the number of channels (in this case 1).

Answer 2

this worked in my case:

inputs = Input(((X_train.shape[1],1)))