Username   Password       Forgot your password?  Forgot your username? 

An Attention-Based Syntax-Tree and Tree-LSTM Model for Sentence Summarization

Volume 13, Number 5, September 2017 - Paper 20  - pp. 775-782
DOI: 10.23940/ijpe.17.05.p20.775782

Wenfeng Liua,b, Peiyu Liua,c,*, Yuzhen Yangb, Yaling Gaod, Jing Yia,e

aSchool of Information Science and Engineering, Shandong Normal University, Jinan, 250014, China
Department of Computer and Information Engineering, Heze University, Heze, 274015, China
Shandong Provincial Key Laboratory for Distributed Computer Software Novel Technology, Jinan,250014, China
Ruipu Peony Industrial Technology Development Co., Ltd, 274015, China
School of Computer Science and Technology, Shandong Jianzhu University, Jinan, 250101, China

(Submitted on January 29, 2017; Revised on April 12, 2017; Accepted on June 23, 20177)


Generative Summarization is of great importance in understanding large-scale textual data. In this work, we propose an attention-based Tree-LSTM model for sentence summarization, which utilizes an attention-based syntactic structure as auxiliary information. Thereinto, block-alignment is used to align the input and output syntax blocks, while inter-alignment is used for alignment of words within that of block pairs. To some extent, block-alignment can prevent structural deviations on the long sentences and inter-alignment is capable of increasing the flexibility of the generation in the blocks. This model can be easily trained to end-to-end mode and deal with any length of the input sentences. Compared with several relatively strong baselines, our model has achieved the state-of-art on DUC-2004 shared task.


References: 21

    1. D. Andor, C. Alberti, D. Weiss, et al., "Globally normalized transition-based neural networks", arXiv preprint arXiv: 1603.06042, 2016.
    2. Ayana, S. Shen, Z. Liu, et al., "Neural headline generation with minimum risk training", arXiv preprint arXiv: 1604.01904, 2016.
    3. D. Bahdanau, K. Cho, Y. Bengio, "Neural machine translation by jointly learning to align and translate", arXiv preprint arXiv: 1409.0473, 2014.
    4. D. Chen, C.D. Manning, "A fast and accurate dependency parser using neural networks", in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 740-750, 2014.
    5. J. Clarke, M. Lapata, "Global inference for sentence compression: An integer linear programming approach", Journal of Artificial Intelligence Research, vol.31, pp. 399-429, 2008.
    6. T. Cohn, M. Lapata, "Sentence compression beyond word deletion", in Proceedings of the International Conference on Computational Linguistics, pp. 137-144, 2008.
    7. K. Filippova, Y. Altun, "Overcoming the lack of parallel data in sentence compression", in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1481-1491, 2013.
    8. K. Knight, D. Marcu, "Summarization beyond sentence extraction: A probabilistic approach to sentence compression", Artificial Intelligence, vol.139, no.1, pp. 91-107, 2002.
    9. P. Koehn, H. Hoang, A. Birch et al., "Moses: Open source toolkit for statistical machine translation", in Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, pp. 177-180, 2007.
    10. K. Lopyrev, "Generating news headlines with recurrent neural networks", arXiv preprint arXiv: 1512.01712, 2015.
    11. C. D. Manning, M. Surdeanu, J. Bauer, et al., "The stanford corenlp natural language processing toolkit", in Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55-60, 2014.
    12. C. D. Manning, P. Raghavan, "Introduction to information retrieval", People's Posts and Telecommunications Press, pp. 824-825, 2010
    13. J. Nivre, "Algorithms for Deterministic Incremental Dependency Parsing", Computational Linguistics, vol.34, no.4, pp. 513-553, 2008.
    14. A. M. Rush, S. Chopra, J. Weston, "A neural attention model for abstractive sentence summarization", arXiv: 1509. 00685v2, 2015.
    15. K. S. Tai, R. Socher, C. D. Manning, "Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks", arXiv preprint arXiv: 1503.00075, 2015.
    16. D. Weiss, C. Alberti, M. Collins, "Structured training for neural network transition-based parsing." arXiv preprint arXiv: 1506.06158, 2015.
    17. K. Woodsend, Y. Feng, M. Lapata, "Generation with quasi-synchronous grammar", in Proceedings of the Association for Computational Linguistics, pp. 513-523, 2010.
    18. S. Wubben, A. V. D Bosch, E. Krahmer, "Sentence simplification by monolingual machine translation", in Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, vol.1, pp. 1015-1024, 2012.
    19. D. Zajic, B. Dorr, R. Schwartz, "Bbn/umd at duc-2004: Topiary", in Proceedings of the HLT-NAACL 2004 Document Understanding Workshop, Boston, pp. 112-119, 2004.
    20. Y. Zhang, J. Nivre, "Transition-based dependency parsing with rich non-local features", in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 740-750, 2011.
    21. H. Zhou, Y. Zhang, S. Huang, et al., "A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing", Association for Computational Linguistics, vol. 1, pp. 1213-1222, 2015



      Click here to download the paper.

      Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

      This site uses encryption for transmitting your passwords.