关键词: 损失函数, 支持向量机, 平方损失函数, 参数选择, VC维

Abstract: Loss function is used to quantify information loss and false degree in regression analysis. This paper addresses heuristic loss function selection for linear regression. For a given noise density, there exists an optimal loss function under an asymptotic setting i.e. squared loss is optimal for Gaussian noise density. However, in reallife applications the noise density is always unknown and the training samples are finite. Robust statistics provides ways for selecting the loss function using statistical information about noise density, however robust statistics is based on asymptotic assumption and may not be well applied for finite sample data sets. For such practical problems, we try to utilize concept of Vapnik’s εinsensitive loss function. We propose a heuristic method for setting the value of ε as a function of samples and noise variance. Experimental comparisons for linear regression problems show that the proposed loss function performs more robustly performance and yields higher prediction accuracy compared with popular squared loss and Huber’s leastmodulus loss. 

Key words: support vector machine, square loss function, parameter selection, VC dimension