YouTube Video Summarizer using NLP: A Review

doi:10.23940/ijpe.23.12.p6.817823

Abstract

Abstract: This review paper delves into the emerging realm of YouTube video summarization utilizing Natural Language Processing (NLP) techniques, a critical area of research with increasing prominence in our multimedia-rich digital age. The paper commences with a broad overview of the field, elaborating on the need for automated video summarization tools to navigate and condense the massive, ever-growing sea of YouTube content. Further, we systematically scrutinize the role and implementation of NLP methods in extracting meaningful textual data from videos, focusing on video transcripts, closed captions, user comments, and associated metadata. Subsequent sections dissect seminal and recent works, studying various NLP techniques such as text summarization, sentiment analysis, topic modeling, and deep learning architectures employed in this context. The paper also focuses on the various metrics used for evaluation and shows datasets generally used to assess the performance of these summarization systems. Finally, we identify current challenges and potential future directions for research in the area, acknowledging the evolving landscape of online video platforms and AI technologies. This review aims to provide researchers and practitioners with an encompassing perspective on the pivotal role of NLP in enabling more efficient, accurate, and intuitive navigation of YouTube content ultimately shaping our digital consumption experiences.

Key words: Natural Language Processing (NLP), YouTube video summarization, text summarization, video content analysis, artificial intelligence

Yogendra Singh, Rishu Kumar, Soumya Kabdal, and Prashant Upadhyay. YouTube Video Summarizer using NLP: A Review [J]. Int J Performability Eng, 2023, 19(12): 817-823.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 43

[1]	Otani, M., Nakashima, Y., Rahtu, E., Heikkil?, J., and Yokoya, N. Video Summarization using Deep Semantic Features. InComputer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part V 13, Springer International Publishing, pp. 361-377, 2017.
[2]	Apostolidis, E., Adamantidou, E., Metsai, A.I., Mezaris, V., and Patras, I. Video Summarization using Deep Neural Networks: A Survey.Proceedings of the IEEE,vol. 109, no. 11, pp. 1838-1863, 2021.
[3]	Zhang, S., Zhu, Y., and Roy-Chowdhury, A.K. Context-Aware Surveillance Video Summarization.IEEE Transactions on Image Processing,vol. 25, no. 11, pp. 5469-5478, 2016.
[4]	Kwon, J. and Lee, K.M. A Unified Framework for Event Summarization and Rare Event Detection from Multiple Views.IEEE transactions on pattern analysis and machine intelligence,vol. 37, no. 9, pp. 1737-1750, 2014.
[5]	Sridevi, M. and Kharde, M. Video Summarization using Highlight Detection and Pairwise Deep Ranking Model.Procedia Computer Science,vol. 167, pp. 1839-1848, 2020.
[6]	Varini, P., Serra, G., and Cucchiara, R. Personalized Egocentric Video Summarization of Cultural Tour on User Preferences Input.IEEE Transactions on Multimedia,vol. 19, no. 12, pp. 2832-2845, 2017.
[7]	Ji, Z., Xiong, K., Pang, Y., and Li, X. Video Summarization with Attention-Based Encoder–Decoder Networks.IEEE Transactions on Circuits and Systems for Video Technology,vol. 30, no. 6, pp. 1709-1717, 2019.
[8]	Fajtl, J., Sokeh, H.S., Argyriou, V., Monekosso, D., and Remagnino, P. Summarizing Videos with Attention. InComputer Vision–ACCV 2018 Workshops: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers 14, Springer International Publishing, pp. 39-54, 2019.
[9]	Gupta, H. and Patel, M. Method of Text Summarization using LSA and Sentence Based Topic Modelling with Bert. In2021 international conference on artificial intelligence and smart systems (ICAIS), IEEE, pp. 511-517, 2021.
[10]	Jugran, S., Kumar, A., Tyagi, B.S., and Anand, V. Extractive Automatic Text Summarization using SpaCy in Python & NLP. In2021 International conference on advance computing and innovative technologies in engineering (ICACITE) IEEE, pp. 582-585, 2021.
[11]	Adhikari, S. Nlp Based Machine Learning Approaches for Text Summarization. In2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), IEEE, pp. 535-538, 2020.
[12]	Madhuri, J.N. and Kumar, R.G. Extractive Text Summarization using Sentence Ranking. In2019 international conference on data science and communication (IconDSC), IEEE, pp. 1-3, 2019.
[13]	Merchant, K. and Pande, Y. Nlp Based Latent Semantic Analysis for Legal Text Summarization. In2018 international conference on advances in computing, communications and informatics (ICACCI), IEEE, pp. 1803-1807, 2018.
[14]	Ngo, C.W., Ma, Y.F., and Zhang, H.J. Video Summarization and Scene Detection by Graph Modeling.IEEE Transactions on circuits and systems for video technology,vol. 15, no. 2, pp. 296-305, 2005.
[15]	Gygli, M., Grabner, H., Riemenschneider, H., and Van Gool, L. Creating Summaries from User Videos. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13, Springer International Publishing, pp. 505-520, 2014.
[16]	Dilawari, A. and Khan, M.U.G. ASoVS: Abstractive Summarization of Video Sequences.IEEE Access,vol. 7, pp. 29253-29263, 2019.
[17]	Sma?li, K., Fohr, D., González-Gallardo, C.E., Grega, M., Janowski, L., Jouvet, D., Komorowski, A., Ko?bia?, A., Langlois, D., Leszczuk, M. and Mella, O. A First Summarization System of a Video in a Target Language. InMultimedia and Network Information Systems: Proceedings of the 11th International Conference MISSI 2018 11, Springer International Publishing, pp. 77-88, 2019.
[18]	Jaiswal, S. and Misra, M. Automatic Indexing of Lecture Videos using Syntactic Similarity Measures. In2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), IEEE, pp. 164-169, 2018.
[19]	Choudhary, P., Munukutla, S.P., Rajesh, K.S., and Shukla, A.S. Real Time Video Summarization on Mobile Platform. In2017 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp. 1045-1050, 2017.
[20]	Kannan, R., Ghinea, G., Swaminathan, S., and Kannaiyan, S. Improving Video Summarization Based on User Preferences. In2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), IEEE, pp. 1-4, 2013.
[21]	Basak, J., Luthra, V., and Chaudhury, S. Video Summarization with Supervised Learning. In2008 19th International Conference on Pattern Recognition, IEEE, pp. 1-4, 2008.
[22]	Huang, J.H., Murn, L., Mrak, M., and Worring, M. Gpt2mvs: Generative Pre-Trained Transformer-2 for Multi-Modal Video Summarization. InProceedings of the 2021 International Conference on Multimedia Retrieval, pp. 580-589, 2021.
[23]	Narasimhan, M., Rohrbach, A., and Darrell, T. Clip-It! Language-Guided Video Summarization.Advances in Neural Information Processing Systems,vol. 34, pp. 13988-14000, 2021.
[24]	Huang, J.H. and Worring, M. Query-Controllable Video Summarization. InProceedings of the 2020 International Conference on Multimedia Retrieval, pp. 242-250, 2020.
[25]	Xiao, S., Zhao, Z., Zhang, Z., Guan, Z., and Cai, D. Query-Biased Self-Attentive Network for Query-Focused Video Summarization.IEEE Transactions on Image Processing,vol. 29, pp. 5889-5899, 2020.
[26]	Nalla, S., Agrawal, M., Kaushal, V., Ramakrishnan, G., and Iyer, R. Watch Hours in Minutes: Summarizing Videos with User Intent. InComputer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, Springer International Publishing, pp. 714-730, 2020.
[27]	Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, ?., and Polosukhin, I. Attention Is All You Need.Advances in neural information processing systems,vol. 30, 2017.
[28]	Jiang, P. and Han, Y. Hierarchical Variational Network for User-Diversified & Query-Focused Video Summarization. InProceedings of the 2019 on International Conference on Multimedia Retrieval, pp. 202-206, 2019.
[29]	Vasudevan, A.B., Gygli, M., Volokitin, A., and Van Gool, L. Query-Adaptive Video Summarization via Quality-Aware Relevance Estimation. InProceedings of the 25th ACM international conference on Multimedia, pp. 582-590, 2017.
[30]	Gygli, M., Grabner, H., and Van Gool, L. Video Summarization by Learning Submodular Mixtures of Objectives. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 3090-3098, 2015.
[31]	Sharghi, A., Laurel, J.S., and Gong, B. Query-Focused Video Summarization: Dataset, Evaluation, and a Memory Network Based Approach. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 4788-4797, 2017.
[32]	Sharghi, A., Gong, B. and Shah, M. Query-Focused Extractive Video Summarization. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, Springer International Publishing, pp.3?19, 2016.
[33]	Zhang, Y., Kampffmeyer, M., Liang, X., Tan, M., and Xing, E.P. Query-Conditioned Three-Player Adversarial Network for Video Summarization.arXiv preprint arXiv:1807.06677, 2018.
[34]	Zhang, Y., Kampffmeyer, M., Zhao, X., and Tan, M. Deep Reinforcement Learning for Query-Conditioned Video Summarization.Applied Sciences,vol. 9, no. 4, pp. 750, 2019.
[35]	Sreeja, M.U. and Kovoor, B.C. A Unified Model for Egocentric Video Summarization: An Instance-Based Approach.Computers & Electrical Engineering,vol. 92, pp. 107161, 2021.
[36]	Ahmed, S.A., Dogra, D.P., Kar, S., Patnaik, R., Lee, S.C., Choi, H., Nam, G.P., and Kim, I.J. Query-Based Video Synopsis for Intelligent Traffic Monitoring Applications.IEEE Transactions on Intelligent Transportation Systems,vol. 21, no. 8, pp. 3457-3468, 2019.
[37]	Gao, J., Yang, X., Zhang, Y., and Xu, C. Unsupervised Video Summarization via Relation-Aware Assignment Learning.IEEE Transactions on Multimedia,vol. 23, pp. 3203-3214, 2020.
[38]	De Avila, S.E.F., Lopes, A.P.B., da Luz Jr, A., and de Albuquerque Araújo, A. VSUMM: A Mechanism Designed to Produce Static Video Summaries and a Novel Evaluation Method.Pattern recognition letters,vol. 32, no. 1, pp. 56-68, 2011.
[39]	Zhang, K., Chao, W.L., Sha, F., and Grauman, K. Video Summarization with Long Short-Term Memory. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14, Springer International Publishing, pp. 766-782, 2016.
[40]	Mahasseni, B., Lam, M., and Todorovic, S. Unsupervised Video Summarization with Adversarial LSTM Networks. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 202-211, 2017.
[41]	Rochan, M., Ye, L., and Wang, Y. Video Summarization using Fully Convolutional Sequence Networks. InProceedings of the European conference on computer vision (ECCV), pp. 347-363, 2018.
[42]	Zhou, K., Qiao, Y., and Xiang, T. Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward. InProceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.
[43]	Ul Haq, H.B., Asif, M., Ahmad, M.B., Ashraf, R., and Mahmood, T. An Effective Video Summarization Framework Based on the Object of Interest using Deep Learning.Mathematical Problems in Engineering,vol. 2022, 2022.