基于隐马尔科夫模型和卷积神经网络的图像标注方法

Automatic Image Annotation Based on Hidden Markov Model and Convolutional Neural Network

摘要: 开发大规模图像库的搜索和浏览算法，使得图像自动标注的重要性日益增强。基于隐马尔科夫模型(HMM)与卷积神经网络(CNN)，我们提出了一种新的图像标注方法HMM + CNN。首先，训练一个多标签学习的CNN网络作为概念分类器；其次，通过一阶HMM模型把图像内容与语义相关性相结合以精炼该CNN的预测分数；最后，为改善对稀疏概念的标注性能，应用梯度下降算法来补偿在真实应用中不平衡图像集上标注概念的频率差。在IAPR TC-12标准图像标注数据集上对比了其他传统方法，结果表明我们的标注方法在查准率和查全率上性能更优。

Abstract: Automatic image annotation is becoming increasingly important in order to develop algorithms that are able to search and browse large-scale image databases. In this paper, we propose a novel annotation approach termed HMM + CNN, which is based on Hidden Markov Model (HMM) and Convolutional Neural Network (CNN). First, a multilabel CNN is trained as a concept classifier. Then, through a first-order HMM, image content and semantics correlation is combined to refine the predicted semantic scores. Finally, to improve the performance of labeling rare concepts, the gradient descent algorithm is applied for compensating the varying frequencies of concepts derived from imbalanced image datasets. Experiments have been carried out on IAPR TC-12 image annotation database. The results show that our proposed approach performs favorably compared with several conventional methods.