'I Need Assistance to Interpret the Score as a means to decide on the best regressor for my ML model
I am working on a Model for Machine Learning and was able to generate the scores of the processes. I am not sure how to use them to make a decision on which is the best kernel to implement for my final prediction or, more specifically, how to reach the best value percentage of the ensemble from a stack.
For example, I am using these functions to generate the scores for each one of my regressors.
n_folds = 5
def rmsle_cv(model):
kf = KFold(n_folds, shuffle=True, random_state=42).get_n_splits(train.values)
rmse= np.sqrt(-cross_val_score(model, train.values, y_train, scoring="neg_mean_squared_error", cv = kf))
return(rmse)
def rmsle(y, y_pred):
return np.sqrt(mean_squared_error(y, y_pred))
Then, I need to build the ensemble to generate my final prediction, utilizing the following two components. My Score Results are posted last. Given the Score results, how should I build the ensemble? To clarify, the stacked_pred is derived from:
stacked_averaged_models = Stacking_Averaged_Models(base_models = (ENet, GBoost, KRR), meta_model = lasso)
stacked_pred = np.expm1(stacked_averaged_models.predict(test.values))
Which is then completed with the following ensemble to generate the final prediction list:
ensemble = stacked_pred*0.4 + xgb_pred*0.2 + lgb_pred*0.2 + XGBReg_pred*0.2
The multiple score's report is posted below and provides the list that I can use to build a better ensemble (I assume). Please provide some guidance that explain the meaning of each score so I can use that understanding to build the ultimate ensemble for my prediction.
[model] Lasso score(cv): 0.1117 (0.0073)
[model] ElasticNet score(cv): 0.1116 (0.0074)
[model] Kernel Ridge score(cv): 0.1158 (0.0075)
[model] Gradient Boosting score(cv): 0.1162 (0.0086)
[model] Xgboost score(cv): 0.1164 (0.0060)
[model] LGBM score(cv): 0.1173 (0.0061)
[model] Averaged base models score(cv): 0.1088 (0.0075)
[model] Stacked averaged-models score(cv): 0.1084 (0.0070)
[model] stacked regressor: 0.0779745304893225
[model] xgboost: 0.07845754741653387
[model] LightGBMs: 0.07201089217729068
[model] XGBReg: 0.05490294827735702
[model] ensemble: 0.0685856065104715
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|