Username   Password       Forgot your password?  Forgot your username? 


Video Indexing and Retrieval based on Key Frame Extraction

Volume 14, Number 8, August 2018, pp. 1824-1832
DOI: 10.23940/ijpe.18.08.p19.18241832

Wenshi Wanga,b, Zhangqin Huanga,b, Weidong Wanga,b, Shuo Zhanga,b, and Rui Tiana,b

aBeijing Advanced Innovation Center for Future Internet Technology, Beijing University of Technology, Beijing, 100124, China
bBeijing Engineering Research Center for IoT Software and Systems, Beijing University of Technology, Beijing, 100124, China

(Submitted on May 1, 2018; Revised on June 13, 2018; Accepted on July 20, 2018)


With the ever growing amount of digital video data becoming available, people are gradually challenged to come up with methods that facilitate video indexing and retrieval. This paper presents a key frame based method that employs shot boundary detection and “bag-of-visual-words (BoW)” based on local keypoints for key frame extraction and semantic concept detection. The performance of BoW features is optimized by choosing appropriate representation choices. Once video frames are represented by BoW features, we can adopt a spectral clustering algorithm for the generation of key frames in each shot, and then we can classify these key frames using support vector machines for video indexing. Finally, this paper performs a query by concept search for video retrieval. The experimental results demonstrate that the proposed approach is capable of retrieving videos. Compared with the existing related method, the proposed method yields better results for key frame extraction and yields a mean average precision (MAP) of 0.68 for the video retrieval model.


References: 21

              1. Y. Lu, C. Shahabi, and S. H. Kim, “Efficient Indexing and Retrieval of Large-scale Geo-tagged Video Databases,” Geoinformatica, Vol. 20, No. 4, pp. 829-857, October 2016
              2. A. Yoshitaka, “Image Video Indexing Retrieval and Summarization based on Eye Movement,” in Proceedings of 4th International Conference on Computing and Informatics (ICOCI 2013), Sarawak, pp. 28-30, Malaysia, August 2013
              3. X. Y. Wei and C. W. Ngo, “Fusing Semantics, Observability, Reliability and Diversity of Concept Detectors for Video Search,” in Proceedings of the 16th International Conference on Multimedia, pp. 26-31, Vancouver, Canada, October 2008
              4. C. L. D. Souza, F. L. C. Pádua, C. F. G. Nunes, G.T. Assis and G. D. Silva, “A Unified Approach to Content-based Indexing and Retrieval of Digital Videos,” Journal of Artificial Intelligence Research, Vol. 3, No. 3, pp. 49-61, 2014
              5. A. K. Mallick and S. Maheshkar, “Video Retrieval based on Color Correlation Histogram Scheme of Clip Segmented Key Frames,” in Proceedings of International Conference on Parallel, Distributed and Grid Computing (PDGC), pp. 213-218, Waknaghat, India, December 2016
              6. D. Saravanan, “Effective Video Data Retrieval using Image Key Frame Selection,” in Proceedings of International Conference on Computational Intelligence and Informatics (ICCII), Hyderabad, India, May 2016
              7. N. E. O’Connor, S. Marlow, N. Murphy, A. F. Smeaton, P. Browne, S. Deasy, H. Lee, and K. McDonald, “Fischlar: An Online System for Indexing and Browsing Broadcast Television Content,” in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Salt Lake City, UT, May 2001
              8. X. Chen, K. Jia, and Z. Deng, “A Video Retrieval Algorithm based on Spatio-temporal Feature Curves and Key Frames,” in Proceedings of Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 287-290, Three Gorges, China, December 2008
              9. S. Memar, L. S. Affendey, N. Mustapha, S. C. Doraisamy, and M. Ektefa, “An Integrated Semantic-based Approach in Concept Based Video Retrieval,” Multimedia Tools and Applications, Vol.64, No. 1, pp. 77-95, May 2013
              10. I. Bartolini, M. Patella, and C. Romani, “SHIATSU: Tagging and Retrieving Videos Without Worries,” Multimedia Tools and Applications, Vol. 63, No. 2, pp. 357-385, March 2013
              11. R. Aly, D. Hiemstra, A. D. Vries, and F. D. Jong, “A Probabilistic Ranking Framework using Unobservable Binary Events for Video Search,” in Proceedings of International Conference on Content-based Image and Video Retrieval, pp. 349-358, Niagara Falls, Canada, July 2008
              12. J. W. Jeong, H. K. Hong, and D. H. Lee, “Ontology-based Automatic Video Annotation Technique in Smart TV Environment,” IEEE Transactions on Consumer Electronics, Vol. 57, No. 4, pp. 1830-1836, 2011
              13. M. S. Zarchi, A. Monadjemi, and K. Jamshidi, “A Concept-based Model for Image Retrieval Systems,” Computers and Electrical Engineering, Vol. 46, pp. 303-313, August 2015
              14. P. Toharia, O. D. Robles, A. F. Smeaton, and A. Rodriguez, “Measuring the Influence of Concept Detection on Video Retrieval,” in Proceedings of International Conference on Computer Analysis of Images and Patterns, Munster, Germany, September 2009
              15. H. Lee, S. Park, and J. H. Yoo, “A Data Cube Model for Surveillance Video Indexing and Retrieval,” in Proceedings of the 10th International Conference on Signal Processing and Multimedia Applications, Reykjavik, Iceland, July 2013
              16. J. Baber, N. Afzulpurkar, and S. Satoh, “A Framework for Video Segmentation using Global and Local Features,” International Journal of Pattern Recognition and Artificial intelligence, Vol. 27, No. 5, August 2013
              17. J. Yang, Y. G. Jiang, A. G. Hauptmann, and C. W. Ngo, “Evaluating Bag-of-visual-words Representations in Scene Classification,” in Proceedings of the International Workshop on Multimedia Information Retrieval, MIR ’07, pp. 197-206, ACM, New York, NY, USA, 2007
              18. Y. G. Jiang, J. Yang, C. W. Ngo, and A. G. Hauptmann, “Representations of Keypoint-based Semantic Concept Detection: A Comprehensive Study,” IEEE Transactions on Multimedia, Vol. 12, No. 1, pp. 42-53, January 2010
              19. V. T. Chasanis, A. C. Likas, and N. P. Galatsanos, “Scene Detection in Videos using Shot Clustering and Sequence Alignment,” IEEE Transactions on Multimedia, Vol. 11, No. 1, pp. 89-100, January 2009
              20. Y. X. Fang and J. H. Wang, “Selection of the Number of Clusters Via the Bootstrap Method,” Computational Statistics and Data Analysis, Vol. 56, No. 3, pp. 468-477, March 2012
              21. C. C. Chang and C. J. Lin, “LIBSVM: A Library for Support Vector Machines,” (, 2001)


                          Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

                          This site uses encryption for transmitting your passwords.