学习曲线(learning curve)来判断模型状态：过拟合欠拟合 - 好文

学习曲线(learning curve)来判断模型状态：过拟合欠拟合

学习曲线是什么

学习曲线是不同训练集大小，模型在训练集和验证集上的得分变化曲线。也就是以样本数为横坐标，训练和交叉验证集上的得分（如准确率）为纵坐标。learning
curve可以帮助我们判断模型现在所处的状态：过拟合（overfiting / high variance） or 欠拟合（underfitting /
high bias）
模型欠拟合、过拟合、偏差和方差平衡时对应的学习曲线如下图所示：

怎么看学习曲线

左上角的图中训练集和验证集上的曲线能够收敛。在训练集合验证集上准确率相差不大，却都很差。这说明模拟对已知数据和未知都不能进行准确的预测，属于高偏差。这种情况模型很可能是欠拟合。可以针对欠拟合采取对应的措施。

右上角的图中模型在训练集上和验证集上的准确率差距很大。说明模型能够很好的拟合已知数据，但是泛化能力很差，属于高方差。模拟很可能过拟合，要采取过拟合对应的措施

怎么画学习曲线

官方学习曲线例子
import numpy as np import matplotlib.pyplot as plt from sklearn.naive_bayes
import GaussianNB from sklearn.svm import SVC from sklearn.datasets import
load_digitsfrom sklearn.model_selection import learning_curve from
sklearn.model_selectionimport ShuffleSplit def plot_learning_curve(estimator,
title, X, y, ylim=None, cv=None, n_jobs=1, train_sizes=np.linspace(.1, 1.0, 5)):
""" 画出data在某模型上的learning curve. 参数解释 ---------- estimator : 你用的分类器。 title :
表格的标题。 X : 输入的feature，numpy类型 y : 输入的target vector ylim : tuple格式的(ymin, ymax),
设定图像中纵坐标的最低点和最高点 cv :
做cross-validation的时候，数据分成的份数，其中一份作为cv集，其余n-1份作为training(默认为3份) n_jobs :
并行的的任务数(默认1) """ plt.figure() plt.title(title) if ylim is not None:
plt.ylim(*ylim) plt.xlabel("Training examples") plt.ylabel("Score")
train_sizes, train_scores, test_scores = learning_curve( estimator, X, y,
cv=cv, n_jobs=n_jobs, train_sizes=train_sizes) train_scores_mean =
np.mean(train_scores, axis=1) train_scores_std = np.std(train_scores, axis=1)
test_scores_mean = np.mean(test_scores, axis=1) test_scores_std =
np.std(test_scores, axis=1) plt.grid() plt.fill_between(train_sizes,
train_scores_mean - train_scores_std, train_scores_mean + train_scores_std,
alpha=0.1, color="r") plt.fill_between(train_sizes, test_scores_mean -
test_scores_std, test_scores_mean + test_scores_std, alpha=0.1, color="g")
plt.plot(train_sizes, train_scores_mean,'o-', color="r", label="Training score"
) plt.plot(train_sizes, test_scores_mean,'o-', color="g", label=
"Cross-validation score") plt.legend(loc="best") plt.draw() plt.show() midpoint
= ((train_scores_mean[-1] + train_scores_std[-1]) + (test_scores_mean[-1] -
test_scores_std[-1])) / 2 diff = (train_scores_mean[-1] + train_scores_std[-1])
- (test_scores_mean[-1] - test_scores_std[-1]) return midpoint, diff digits =
load_digits() X, y = digits.data, digits.target title ="Learning Curves (Naive
Bayes)" # Cross validation with 100 iterations to get smoother mean test and
train # score curves, each time with 20% data randomly selected as a validation
set. cv = ShuffleSplit(n_splits=100, test_size=0.2, random_state=0) estimator =
GaussianNB() plot_learning_curve(estimator, title, X, y, ylim=(0.7, 1.01),
cv=cv, n_jobs=4) title = "Learning Curves (SVM, RBF kernel, $\gamma=0.001$)" #
SVC is more expensive so we do a lower number of CV iterations: cv =
ShuffleSplit(n_splits=10, test_size=0.2, random_state=0) estimator = SVC(gamma=
0.001) plot_learning_curve(estimator, title, X, y, (0.7, 1.01), cv=cv, n_jobs=4)
参考文献

用学习曲线 learning curve 来判别过拟合问题
<http://blog.csdn.net/aliceyangxi1987/article/details/73598857>
机器学习系统模型调优实战–所有调优技术都附相应的scikit-learn实现
<http://blog.csdn.net/xlinsist/article/details/51344449>
官方文档
<http://scikit-learn.org/stable/auto_examples/model_selection/plot_learning_curve.html>

热门工具换一换