An Attention-Based Syntax-Tree and Tree-LSTM Model for Sentence Summarization

doi:10.23940/ijpe.17.05.p20.775782

Abstract

Abstract: Generative Summarization is of great importance in understanding large-scale textual data. In this work, we propose an attention-based Tree-LSTM model for sentence summarization, which utilizes an attention-based syntactic structure as auxiliary information. Thereinto, block-alignment is used to align the input and output syntax blocks, while inter-alignment is used for alignment of words within that of block pairs. To some extent, block-alignment can prevent structural deviations on the long sentences and inter-alignment is capable of increasing the flexibility of the generation in the blocks. This model can be easily trained to end-to-end mode and deal with any length of the input sentences. Compared with several relatively strong baselines, our model has achieved the state-of-art on DUC-2004 shared task.

Submitted on January 29, 2017; Revised on April 12, 2017; Accepted on June 23, 20177
References: 21

Wenfeng Liu, Peiyu Liu, Yuzhen Yang, Yaling Gao, and Jing Yi. An Attention-Based Syntax-Tree and Tree-LSTM Model for Sentence Summarization [J]. Int J Performability Eng, 2017, 13(5): 775-782.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 0

	D. Andor, C. Alberti, D. Weiss, et al., "Globally normalized transition-based neural networks", arXiv preprint arXiv: 1603.06042, 2016.
	Ayana, S. Shen, Z. Liu, et al., "Neural headline generation with minimum risk training", arXiv preprint arXiv: 1604.01904, 2016.
	D. Bahdanau, K. Cho, Y. Bengio, "Neural machine translation by jointly learning to align and translate", arXiv preprint arXiv: 1409.0473, 2014.
	D. Chen, C.D. Manning, "A fast and accurate dependency parser using neural networks", in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 740-750, 2014.
	J. Clarke, M. Lapata, "Global inference for sentence compression: An integer linear programming approach", Journal of Artificial Intelligence Research, vol.31, pp. 399-429, 2008.
	T. Cohn, M. Lapata, "Sentence compression beyond word deletion", in Proceedings of the International Conference on Computational Linguistics, pp. 137-144, 2008.
	K. Filippova, Y. Altun, "Overcoming the lack of parallel data in sentence compression", in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1481-1491, 2013.
	K. Knight, D. Marcu, "Summarization beyond sentence extraction: A probabilistic approach to sentence compression", Artificial Intelligence, vol.139, no.1, pp. 91-107, 2002.
	P. Koehn, H. Hoang, A. Birch et al., "Moses: Open source toolkit for statistical machine translation", in Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, pp. 177-180, 2007.
	K. Lopyrev, "Generating news headlines with recurrent neural networks", arXiv preprint arXiv: 1512.01712, 2015.
	C. D. Manning, M. Surdeanu, J. Bauer, et al., "The stanford corenlp natural language processing toolkit", in Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55-60, 2014.
	C. D. Manning, P. Raghavan, "Introduction to information retrieval", People's Posts and Telecommunications Press, pp. 824-825, 2010
	J. Nivre, "Algorithms for Deterministic Incremental Dependency Parsing", Computational Linguistics, vol.34, no.4, pp. 513-553, 2008.
	A. M. Rush, S. Chopra, J. Weston, "A neural attention model for abstractive sentence summarization", arXiv: 1509. 00685v2, 2015.
	K. S. Tai, R. Socher, C. D. Manning, "Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks", arXiv preprint arXiv: 1503.00075, 2015.
	D. Weiss, C. Alberti, M. Collins, "Structured training for neural network transition-based parsing." arXiv preprint arXiv: 1506.06158, 2015.
	K. Woodsend, Y. Feng, M. Lapata, "Generation with quasi-synchronous grammar", in Proceedings of the Association for Computational Linguistics, pp. 513-523, 2010.
	S. Wubben, A. V. D Bosch, E. Krahmer, "Sentence simplification by monolingual machine translation", in Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, vol.1, pp. 1015-1024, 2012.
	D. Zajic, B. Dorr, R. Schwartz, "Bbn/umd at duc-2004: Topiary", in Proceedings of the HLT-NAACL 2004 Document Understanding Workshop, Boston, pp. 112-119, 2004.
	Y. Zhang, J. Nivre, "Transition-based dependency parsing with rich non-local features", in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 740-750, 2011.
	H. Zhou, Y. Zhang, S. Huang, et al., "A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing", Association for Computational Linguistics, vol. 1, pp. 1213-1222, 2015