星期日, 九月 25, 2011

caret包应用之四:模型预测与检验

模型建立好后,我们可以利用predict函数进行预测,例如预测检测样本的前五个
predict(gbmFit1, newdata = testx)[1:5]
为了比较不同的模型,还可用装袋决策树建立第二个模型,命名为gbmFit2
gbmFit2= train(trainx, trainy,method = "treebag",trControl = fitControl)
models = list(gbmFit1, gbmFit2)
另一种得到预测结果的方法是使用extractPrediction函数,得到的部分结果如下显示

predValues = extractPrediction(models,testX = testx, testY = testy)
head(predValues)

     obs     pred model dataType  object
1 Active   Active   gbm Training Object1
2 Active   Active   gbm Training Object1
3 Active Inactive   gbm Training Object1
4 Active   Active   gbm Training Object1
5 Active   Active   gbm Training Object1 
6 Active Active gbm Training Object1

从中可提取检验样本的预测结果
testValues = subset(predValues, dataType == "Test")
如果要得到预测概率,则使用extractProb函数
probValues = extractProb(models,testX = testx, testY = testy)
testProbs = subset(probValues, dataType == "Test")
对于分类问题的效能检验,最重要的是观察预测结果的混淆矩阵
Pred1 = subset(testValues, model == "gbm")
Pred2 = subset(testValues, model == "treebag")
confusionMatrix(Pred1$pred, Pred1$obs)
confusionMatrix(Pred2$pred, Pred2$obs)
结果如下,可见第一个模型在准确率要比第二个模型略好一些

          Reference
Prediction Active Inactive
  Active       65       12
  Inactive      9       45
                                      
               Accuracy : 0.8397          


          Reference
Prediction Active Inactive
  Active       63       12
  Inactive     11       45
                                          
               Accuracy : 0.8244          

最后是利用ROCR包来绘制ROC图
prob1 = subset(testProbs, model == "gbm")
prob2 = subset(testProbs, model == "treebag")
library(ROCR)
prob1$lable=ifelse(prob1$obs=='Active',yes=1,0)
pred1 = prediction(prob1$Active,prob1$lable)
perf1 = performance(pred1, measure="tpr", x.measure="fpr" )
plot( perf1 )


没有评论:

发表评论