Short Text Classification based on Feature Extension using Information in Images

doi:10.23940/ijpe.19.02.p31.667675

Abstract

Abstract: With the quick development and extensive application of the Internet, there is a growing desire for people to share their life or opinions on social networks, which producesa mass of short texts. Short texts are characterized by short length, sparse features, and a lack of contextual information. Thus, it is difficultfor conventional methods to achieve high quality classification performance. To achieve a higher classification accuracy, this paper proposes a novel short text classification method based on feature extension by incorporating the information of the images.Specifically, we first generate a sentence that descripts the images by image caption technology, and then we combine the generated sentence with the text as the input of the classifier. Meanwhile, we introduce a similarity module in terms of the correlation between the image and the short text so as to determine whether the two sentences are combined or not. Simulation results show that ourproposed model significantly out performs the state-of-the-art methods in terms of classification accuracy.

Key words: short text classification, image caption, feature extension, sentence similarity

Shengjie Zhao and Qianyun Jiang. Short Text Classification based on Feature Extension using Information in Images [J]. Int J Performability Eng, 2019, 15(2): 667-675.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

Figures/Tables 8

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Table 1

Figure 6

Table 2

References 28

[1]	B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, M. Demirbas , “Short Text Classification in Twitter to Improve Information Filtering,” in Proceedings ofInternational ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 841-842, 2010 doi: 10.1145/1835449.1835643
[2]	M. Wang, L. Lin, F. Wang , “Improving Short Text Classification through Better Feature Space Selection,” in Proceedings of International Conference on Computational Intelligence and Security, pp. 120-124, 2014
[3]	D. Bollegala, M. Ishizuka, Y. Matsuo , “Measuring Semantic Similarity Between Words using Web Search Engines,” in Proceedings ofInternational Conference on World Wide Web,pp. 757-766,Banff, Alberta, Canada, May 2007 doi: 10.1145/1242572.1242675
[4]	X. Hu, N. Sun, C. Zhang, T. S. Chua , “Exploiting Internal and External Semantics for the Clustering of Short Texts using World Knowledge,” in Proceedings of ACM Conference on Information and Knowledge Management, pp. 919-928, 2009 doi: 10.1145/1645953.1646071
[5]	A. Paccanaro and G. E. Hinton, “Learning Distributed Representations of Concepts using Linear Relational Embedding,” IEEE Transactions on Knowledge & Data Engineering,Vol. 13,No. 2, pp. 232-244, 2002 doi: 10.1109/69.917563
[6]	X. Zhang and B. Wu, “Short Text Classification based on Feature Extension using the n-Gram Model,” in Proceedings of 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD),pp.710-716, IEEE, 2015 doi: 10.1109/FSKD.2015.7382029
[7]	Christopher Bonnett, “Classifying E-Commerce Products based on Images and Text,” ( http://cbonnett.github.io/Insight.html)
[8]	T. Mikolov, I. Sutskever, K. Chen, G. Corrado, J. Dean , “Distributed Representations of Words and Phrases and their Compositionality,” in Proceedings of International Conference on Neural Information Processing Systems, pp. 3111-3119, 2013
[9]	J. Pennington, R. Socher, C. Manning , “Glove: Global Vectors for Word Representation,” in Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014
[10]	D. M. Blei, A. Y. Ng, M. I. Jordan , “Latent Dirichlet Allocation,” J Machine Learning Research Archive,Vol. 3, pp. 993-1022, 2003
[11]	A. Bordes, J. Weston, R. Collobert, Y. Bengio , “Learning Structured Embeddings of Knowledge Bases,” AAAI, Vol. 6. No. 1, 2011
[12]	R. Johnson and T. Zhang, “Effective Use of Word Order for Text Categorization with Convolutional Neural Networks,” arXiv preprint arXiv:1412.1058, 2014
[13]	Y. Kim , “Convolutional Neural Networks for Sentence Classification,” arXiv preprint arXiv:1408.5882, 2014 doi: 10.3115/v1/D14-1181
[14]	C. N. D . Santos and M. Gattit, “Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts,” in Proceedings of International Conference on Computational Linguistics, 2014
[15]	R. Kiros, R. Salakhutdinov, R. Zemel , “Multimodal Neural Language Models,” in Proceedings of International Conference on Machine Learning, pp. 595-603, 2014
[16]	K. Cho, B. Van Merriënboer, C Gulcehre , et al., “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation,” arXiv preprint arXiv:1406.1078, 2014 doi: 10.3115/v1/D14-1179
[17]	S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation,Vol. 9, No. 8, pp. 1735-1780, 1997
[18]	O. Vinyals, A. Toshev, S. Bengio, D. Erhan , “Show and Tell: A Neural Image Caption Generator,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156-3164, 2015 doi: 10.1109/CVPR.2015.7298935
[19]	O. Vinyals, A. Toshev, S. Bengio, andD. Erhan. “Show and Tell: Lessons Learned fromthe 2015 MSCOCO Image Captioning Challenge,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 652-663, 2016
[20]	M. Surdeanu, M. Ciaramita, H. Zaragoza , “Learning to Rank Answers to Non-Factoid Questions from Web Collections,” Computational Linguistics,Vol. 37, No. 2, pp. 351-383, 2011
[21]	N. Kalchbrenner, E. Grefenstette, P. Blunsom , “A Convolutional Neural Network for Modelling Sentences,” arXiv preprint arXiv:1404.2188, 2014
[22]	A. Severyn and A. Moschitti. “Modeling Relational Information in Question-Answer Pairs with Convolutional Neural Networks,” arXiv preprint arXiv:1604.01178, 2016
[23]	O. Abdel-Hamid, A. R. Mohamed, H. Jiang, G. Penn , “Applying Convolutional Neural Networks Concepts to Hybrid NN-HMM Model for Speech Recognition,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4277-4280, 2012 doi: 10.1109/ICASSP.2012.6288864
[24]	C. Szegedy, L. Wei, J. Yangqing , et al., “Going Deeper with Convolutions,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9, 2015
[25]	W. Zaremba, I. Sutskever, O. Vinyals , “Recurrent Neural Network Regularization,” Eprint Arxiv, 2014
[26]	L. M. Surhone, M. T. Tennoe, S. F. Henssonow , “Long Short Term Memory, ” Betascript Publishing, 2010
[27]	A. Severyn and A. Moschitti, “Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks,” in Proceedings of the International ACM SIGIR Conference, pp. 373-382, 2015
[28]	T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, et al., “Microsoft coco: Common Objects in Context,” in Proceedings of European Conference onComputer Vision, Springer, pp. 740-755, 2014