How to pass a trained model to KerasClassifier?

Question

I have a dozen pre-trained DNNs that I wish to add to a sklearn ensemble. The issue is that it seems I can not provide pre-trained models to KerasClassifier.

classifier_models = []
# models: dict of pre-trained models.
for name, model in models: 
    try:
        # Normal Sklearn models. No need to modify.
        model._estimator_type
        classifier_models.append((name, model))
    except:
        # Pre-trained DNNs (keras) must be wrapped.
        new_model = KerasClassifier(model=model)
        # Standard procedure.
        new_model._estimator_type = 'classifier'
        classifier_models.append((name, new_model))

ERROR:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11744/990400553.py in <module>
      3     try:
----> 4         model._estimator_type
      5         classifier_models.append((name, model))

AttributeError: 'Sequential' object has no attribute '_estimator_type'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11744/990400553.py in <module>
      5         classifier_models.append((name, model))
      6     except:
----> 7         new_model = KerasClassifier(model=model)
      8         new_model._estimator_type = 'classifier'
      9         classifier_models.append((name, new_model))

~\miniconda3\envs\epfl-ml\lib\site-packages\tensorflow\python\keras\wrappers\scikit_learn.py in __init__(self, build_fn, **sk_params)
     75     self.build_fn = build_fn
     76     self.sk_params = sk_params
---> 77     self.check_params(sk_params)
     78 
     79   def check_params(self, params):

~\miniconda3\envs\epfl-ml\lib\site-packages\tensorflow\python\keras\wrappers\scikit_learn.py in check_params(self, params)
     91     ]
     92     if self.build_fn is None:
---> 93       legal_params_fns.append(self.__call__)
     94     elif (not isinstance(self.build_fn, types.FunctionType) and
     95           not isinstance(self.build_fn, types.MethodType)):

AttributeError: 'KerasClassifier' object has no attribute '__call__'

I do not wish to use KerasClassifier with a build function (example: KerasClassifier(build_fn=build_dnn()) because I already have a trained network and it would take a lot of time to re-train.

Answer 1

Apparently, the best way to go about this is using SciKeras which is an enhanced fork of that wrapper class that enables a lot more functionality including functional model types and multi-output plus you can use pre-built models. I'm currently figuring out the same thing you are but this looks promising.

It looks like the first variable model for the scikeras.wrappers.KerasClassifier wrapper can be either a function that builds a keras model or an already built one.

from the scikeras documentation for the KerasClassifier constructor parameters:

model:Union[None, Callable[…, tf.keras.Model], tf.keras.Model],default None

Used to build the Keras Model. When called, **must return a compiled instance of a Keras Model** to be used by fit, predict, etc. If None, you must implement _keras_build_fn.

Additionally:

Attributes model_tf.keras.Model

The instantiated and compiled Keras Model. For pre-built models, this will just be a reference to the passed Model instance.

https://www.adriangb.com/scikeras/stable/generated/scikeras.wrappers.KerasClassifier.html

So for example your wrapper would look like so:

wrapped_model = KerasClassifier(model=pre_built_model_here,optimizer='Adam',...ect)