k-Folds Cross Validation
I personally like this method the most and feel like it is somewhere between two extremes (LOOCV and Validation Set.) Observations will be split into k groups. For example, given that mtcars has 32 observations, and k is 10, eight observations will be randomly assigned to each group (or k.) Then it will work just like LOOCV in which the first group is the test set, then the other nine groups are training set, and so on. Finally, the statistics in each iteration are averaged.
k-Folds still has randomness. However, it is relatively lower than the Validation Set as the final statistics will be an averaged of the k models. In term of speed, it certainly faster than LOOCV as the models will be fit only k times. Yep, it is somewhere between LOOCV and Validation Set. 🙂
As usual, the caret package can quickly perform k-Folds.
|
1 2 3 4 5 |
#First, we define the control. control <- trainControl(method="cv", number = 10) #Train the model k_model<- train(mpg ~., data=mtcars, trControl=control, method="lm") |
|
1 2 |
k_model$result k_model$finalModel |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
> k_model$finalModel Call: lm(formula = .outcome ~ ., data = dat) Coefficients: (Intercept) cyl disp hp drat wt qsec vs 12.30337 -0.11144 0.01334 -0.02148 0.78711 -3.71530 0.82104 0.31776 am gear carb 2.52023 0.65541 -0.19942 > k_model$result intercept RMSE Rsquared RMSESD RsquaredSD 1 TRUE 3.09798 0.8653214 0.9350881 0.1314003 |