How to perform Recursive Feature Elimination (RFE) with LSTM as estimator in python?

Question

I am trying to identify the important features in a data frame containing stock data. I plan on using LSTM to predict closing prices later on. I currently have the following:

df_stockData = pdr.DataReader(ticker, data_source='yahoo', start='2012-10-20',
                              end=str(date.today() - timedelta(days=1)))

def performRFE():
    rfecv = RFECV(
        estimator=RandomForestRegressor(), #I want to use a LSTM instead
        min_features_to_select=1,
        step=2,
        n_jobs=-1,
        scoring="r2",
        cv=5,
    )
    cols_rfe = list(df_stockData.columns)
    cols_rfe.remove('Close')
    rfecv.fit(StandardScaler().fit_transform(df_stockData.loc[:, cols_rfe]),
              np.array(df_stockData['Close'].values).reshape(-1, ))
    return list(df_stockData.loc[:, cols_rfe].columns[rfecv.support_])

selected_features = performRFE()
print("Selected Features: {}".format(selected_features))

As you can see above, I get the data from yahoo finance and I try to find important features which can help me predict the close price. Here I use RandomForestRegressor() as estimator. Instead of this I want to use LSTM because later on I use LSTM to predict the close price. Could anyone help me with this? I have been unable to figure it out yet.

Answer 1

I tried doing the exact same thing but from what I have learned it is not possible to do with LSTM in sklearn. The closest you can get is doing RFE with MLP. Maybe you have found some workaround to this problem by now, if so please share your experience I'm quite interested.