A Survey of the Analysis of Complex Systems based on Complex Network Theory and Deep Learning

doi:10.23940/ijpe.22.04.p2.241250

Abstract

Abstract: From the perspective of complex network theory, complex systems can be characterized by the interaction of microscopic units through nonlinear effects, yielding macroscopic emergent behavior. In light of the powerful capability of deep learning in feature extraction and model fitting from large amount of datasets, we try to overview the benefits of combining the complex network analysis with deep learning techniques to investigate complex systems. We first explore the existence of complexity in complex systems. In what followed, we first give a brief description of complex network theory. Then, we present an overview of deep learning technology. Subsequently, we focus on the research advances and applications in the analysis of complex systems based on complex network theory and deep learning. The last section is further discussion and prospects for the combination of these two methods. In a nutshell, the development of deep learning combined with complex network theory allows for exploring the complexity in complex systems at a higher level.

Key words: complexity, complex systems, complex networks, deep learning

Dan Lu and Shunkun Yang*. A Survey of the Analysis of Complex Systems based on Complex Network Theory and Deep Learning [J]. Int J Performability Eng, 2022, 18(4): 241-250.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. R. Gallagher, T. Appenzeller, D. Normile, et al., “Beyond reductionism,” Science, vol. 284, no.5411, p. 79, 1999.
2. M. M. Waldrop, Complexity: The emerging science at the edge of order and chaos. Simon and Schuster, 1993.
3. D. J.Watts and S. H. Strogatz, “Collective dynamics of ‘small-world’ networks,” Nature, vol. 393, pp. 440-442, jun 1998.
4. A.-L. Barabási and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, pp. 509-512, oct 1999.
5. A. Barrat, M. Barthelemy,A. Vespignani, Dynamical processes on complex networks. Cambridge university press, 2008.
6. R. Interdonato, M. Atzmueller, S. Gaito, R. Kanawati, C. Largeron,A. Sala, “Feature-rich networks: going beyond complex network topologies,” Applied Network Science, vol. 4, jan 2019.
7. T. C.Silva and L. Zhao, Machine learning in complex networks, vol. 1. Springer, 2016.
8. G. E.Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, pp. 504-507, jul 2006.
9. J. Louth, “From newton to newtonianism: Reductionism and the development of the social sciences,” Emergence: Complexity and Organization, vol. 13, no.4, p. 63, 2011.
10. J. Earman, “Laplacian determinism, or is this any way to run a universe?,” The Journal of Philosophy, vol. 68, no. 21, pp. 729-744, 1971.
11. F. Mazzocchi, “Complexity in biology: exceeding the limits of reductionism and determinism using complexity theory,” EMBO reports, vol. 9, no. 1, pp. 10-14, 2008.
12. P. W. Anderson, “More is different,” Science, vol. 177, no. 4047, pp. 393-396, 1972.
13. L. da F. Costa, F. A. Rodrigues, G. Travieso, and P. R. V. Boas, “Characterization of complex networks: A survey of measurements,” Advances in Physics, vol. 56, pp. 167-242, jan 2007.
14. R. Cohen and S. Havlin, Complex networks: structure, robustness and function. Cambridge university press, 2010.
15. L. d. F. Costa, O. N. Oliveira Jr, G. Travieso, F. A. Rodrigues, P. R. Villas Boas, L. Antiqueira, M. P. Viana, and L. E. Correa Rocha, “Analyzing and modeling real-world phenomena with complex networks: a survey of applications,” Advances in Physics, vol. 60, no. 3, pp. 329-412, 2011.
16. B. R. Jasny, L. M. Zahn,E. Marshall, “Special issue on complex systems and networks,” Science, vol. 325, no. 5939, pp. 405-432, 2009.
17. L. Euler, “Solutio problematis ad geometriam situs pertinentis,” Commentarii academiae scientiarum Petropolitanae, pp. 128-140, 1741.
18. N. Biggs, E. K. Lloyd,R. J. Wilson, Graph Theory, 1736-1936. Oxford University Press, 1986.
19. A.-L. Barabási, “Network science,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 371, no.1987, p. 20120375, 2013.
20. F. Menczer, S. Fortunato,C. A. Davis, A First Course in Network Science. Cambridge University Press, jan 2020.
21. M. Newman, Networks. Oxford university press, 2018.
22. M. E. Newman, “The structure and function of complex networks,” SIAM review, vol. 45, no. 2, pp. 167-256, 2003.
23. M. Newman, Networks: An Introduction. Oxford University Press, mar 2010. 14
24. A.-L. Barabási, “The network takeover,” Nature Physics, vol. 8, pp. 14-16, dec 2011.
25. Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabasi, “Observability of complex systems,” Proceedings of the National Academy of Sciences, vol. 110, pp. 2460-2465, jan 2013.
26. L. Lü, C.-H. Jin, and T. Zhou, “Similarity index based on local paths for link prediction of complex networks,” Physical Review E, vol. 80, oct 2009.
27. L. Weng, F. Menczer,Y.-Y. Ahn, “Virality prediction and community structure in social networks,” Scientific Reports, vol. 3, aug 2013.
28. L. Zhou, S. Pan, J. Wang,A. V. Vasilakos, “Machine learning on big data: Opportunities and challenges,” Neurocomputing, vol. 237, pp. 350-361, 2017.
29. A. K.Jain and R. C. Dubes, Algorithms for clustering data. Prentice-Hall, Inc., 1988.
30. P. Domingos, “A few useful things to know about machine learning,” Communications of the ACM, vol. 55, no. 10, pp. 78-87, 2012.
31. M. I.Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives, and prospects,” Science, vol. 349, no. 6245, pp. 255-260, 2015.
32. Y. LeCun, Y. Bengio,G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436-444, may 2015.
33. Y. Bengio, P. Lamblin, D. Popovici,H. Larochelle, “Greedy layer-wise training of deep networks,” Advances in neural information processing systems, vol. 19, 2006.
34. I. J. Goodfellow, A. Courville,Y. Bengio, “Scaling up spike-and-slab models for unsupervised feature learning,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1902-1914, 2012.
35. A. Krizhevsky, I. Sutskever,G. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, vol. 25, no. 2, 2012.
36. M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997. 15
37. R. Collobert and J. Weston, “A unified architecture for natural language processing: Deep neural networks with multitask learning,” in Proceedings of the 25th international conference on Machine learning, pp. 160-167, 2008.
38. K. He, X. Zhang, S. Ren,J. Sun, “Deep residual learning for image recognition,” IEEE, 2016.
39. C. Tang, Y. Ling, X. Yang, W. Jin,C. Zheng, “Multi-view object detection based on deep learning,” Applied Sciences, vol. 8, no. 9, 2018.
40. K. Hornik, M. Stinchcombe,H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, pp. 359-366, jan 1989.
41. C.-H. Chang, “Deep and shallow architecture of multilayer neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, pp. 2477-2486, oct 2015.
42. J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks, vol. 61, pp. 85-117, jan 2015.
43. M. Pereda and E. Estrada, “Machine learning analysis of complex networks in hyperspherical space,” arXiv preprint arXiv:1804.05960, 2018.
44. A. A. Margolin, I. Nemenman, K. Basso, C. Wiggins, G. Stolovitzky, R. D. Favera,A. Califano, “ARACNE: An algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context,” BMC Bioinformatics, vol. 7, mar 2006.
45. C. Fan, L. Zeng, Y. Sun,Y.-Y. Liu, “Finding key players in complex networks through deep reinforcement learning,” Nature machine intelligence, vol. 2, no. 6, pp. 317-324, 2020.
46. Z. Gao, W. Dang, X. Wang, X. Hong, L. Hou, K. Ma,M. Perc, “Complex networks and deep learning for EEG signal analysis,” Cognitive Neurodynamics, vol. 15, pp. 369-388, aug, 2020.
47. S. Ha and H. Jeong, “Unraveling hidden interactions in complex systems with deep learning,” Scientific reports, vol. 11, no. 1, pp. 1-13, 2021. 16
48. B. Hussain, Q. Du, S. Zhang, A. Imran,M. A. Imran, “Mobile edge computing-based data-driven deep learning framework for anomaly detection,” IEEE Access, vol. 7, pp. 137656-137667, 2019.
49. Z. Lv, A. K. Singh,J. Li, “Deep learning for security problems in 5g heterogeneous networks,” IEEE Network, vol. 35, pp. 67-73, mar 2021.
50. M. Dhilber,S. D. Bhavani.“Community detection in social networks using deep learning." International conference on distributed computing and internet technology”. Springer, Cham, 2020.
51. J. Cao, et al.“Incorporating network structure with node contents for community detection on large networks using deep learning.” Neurocomputing vol. 297, pp.71-81, 2018.
52. S. Li, et al.“A weighted network community detection algorithm based on deep learning.” Applied Mathematics and Computation, vol. 401, pp. 126012, 2021.
53. Y. Zhang, Y. Xiong, Y. Ye, T. Liu, W. Wang, Y. Zhu,P. S. Yu, “SEAL: Learning heuristics for community detection with generative adversarial networks.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1103-1113, aug, 2020.
54. J. Chenet al., “E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction,” in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 6, pp. 3699-3712, june 2021.
55. H. Wang, X. Shi,D.Y. Yeung, “Relational deep learning: A deep latent variable model for link prediction.” In Thirty-first AAAI conference on artificial intelligence, feb, 2017.
56. C. Chiu and J.Zhan, “Deep Learning for Link Prediction in Dynamic Networks Using Weak Estimators,” in IEEE Access, vol. 6, pp. 35937-35945, 2018.
57. T. C.Silva and L. Zhao, “Network-based high level data classification,” IEEE Transactions on Neural Networks and Learning Systems, vol. 23, pp. 954-970, jun 2012.
58. T. C.Silva and L. Zhao, “High-level pattern-based classification via tourist walks in networks,” Information Sciences, vol. 294, pp. 109-126, feb 2015.
59. A. Celikyilmaz and D. Hakkani-Tur, “A graph-based semi-supervised learning for question semantic labeling,” in Proceedings of the NAACL HLT2010 Workshop on Semantic Search, pp. 27-35, 2010.
60. T. H. Cupertino, J. Huertas,L. Zhao, “Data clustering using controlled consensus in complex networks,” Neurocomputing, vol. 118, pp. 132-140, oct 2013.
61. T. C.Silva and L. Zhao, “Pixel clustering by using complex network community detection technique,” in Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007), IEEE, oct 2007.
62. P. Hamel and D. Eck, “Learning features from music audio with deep belief networks.,” in ISMIR, vol. 10, pp. 339-344, Citeseer, 2010.
63. Y. Bengio, F. Bastien, A. Bergeron, N. Boulanger-Lewandowski, T. Breuel, Y. Chherawala, M. Cisse, M. Côté, D. Erhan, J. Eustache, et al., “Deep learners benefit more from out-of-distribution examples,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 164-172, JMLR Workshop and Conference Proceedings, 2011.
64. J. Susskind, G. Hinton, R. Memisevic,M. Pollefeys, “Modeling the joint density of two images under a variety of transformations,” in CVPR 2011, pp. 2793-2800, IEEE, 2011. 17
65. P. Luo, X. Wang,X. Tang, “Hierarchical face parsing via deep learning,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2480-2487, IEEE, 2012.
66. S. Thomas, M. L. Seltzer, K. Church,H. Hermansky, “Deep neural network features and semi-supervised training for low resource speech recognition,” in 2013 IEEE international conference on acoustics, speech and signal processing, pp. 6704-6708, IEEE, 2013.
67. T. Mikolov, K. Chen, G. Corrado,J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781, 2013.
68. M. Defferrard, X. Bresson,P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” Advances in neural information processing systems, vol. 29, 2016.
69. S. Zhang, H. Tong, J. Xu,R. Maciejewski, “Graph convolutional networks: a comprehensive review,” Computational Social Networks, vol. 6, nov 2019.
70. J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li,M. Sun, “Graph neural networks: A review of methods and applications,” AI Open, vol. 1, pp. 57-81, 2020.
71. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.
72. R. Levie, F. Monti, X. Bresson,M. M. Bronstein, “Cayleynets: Graph convolutional neural networks with complex rational spectral filters,” IEEE Transactions on Signal Processing, vol. 67, no. 1, pp. 97-109, 2018.
73. M. Henaff, J. Bruna,Y. LeCun, “Deep convolutional networks on graph-structured data,” arXiv preprint arXiv:1506.05163, 2015.
74. M. Defferrard, X. Bresson,P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” Advances in neural information processing systems, vol. 29, 2016.
75. T. Nguyen and R. Grishman, “Graph convolutional networks with argument-aware pooling for event detection,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018. 18
76. X. Liu, Z. Luo,H. Huang, “Jointly multiple events extraction via attention-based graph information aggregation,” arXiv preprint arXiv:1809.09078, 2018.
77. L. Yao, C. Mao,Y. Luo, “Graph convolutional networks for text classification,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 7370-7377, 2019.
78. V. Garcia and J. Bruna, “Few-shot learning with graph neural networks,” arXiv preprint arXiv:1711.04043, 2017.
79. S. Zhang, Y. Qin, K. Sun,Y. Lin, “Few-shot audio classification with attentional graph neural networks.,” in Interspeech, pp. 3649-3653, 2019.
80. K. Marino, R. Salakhutdinov,A. Gupta, “The more you know: Using knowledge graphs for image classification,” arXiv preprint arXiv:1612.04844, 2016.
81. X. Li, X. Yan, Q. Gu, H. Zhou, D. Wu,J. Xu, “Deepchemstable: chemical stability prediction with an attention-based graph convolution network,” Journal of chemical information and modeling, vol. 59, no. 3, pp. 1044-1049, 2019.
82. A. Fout, J. Byrd, B. Shariat,A. Ben-Hur, “Protein interface prediction using graph convolutional networks,” Advances in neural information processing systems, vol. 30, 2017.
83. D. Mrowca, C. Zhuang, E. Wang, N. Haber, L. F.Fei-Fei, J.Tenenbaum, and D. L. Yamins, “Flexible neural representation for physics prediction,” Advances in neural information processing systems, vol. 31, 2018.
84. T. Kipf, E. Fetaya, K. C. Wang, M. Welling,R. Zemel, “Neural relational inference for interacting systems.” In International Conference on Machine Learning, pp. 2688-2697, PMLR, july, 2018.
85. P. Mehta and D. J. Schwab, “An exact mapping between the variational renormalization group and deep learning,” arXiv prepr

[1]	Pancham Singh, Updesh Kumar Jaiswal, Eshank Jain, Nikhil Kumar, and Vimlesh Mishra. A Novel Methodology Utilizing Modern CCTV Cameras and Software as a Service Model for Crime Detection and Prediction [J]. Int J Performability Eng, 2025, 21(2): 112-121.
[2]	Seema Kalonia and Amrita Upadhyay. Comparative Analysis of Machine Learning Model and PSO Optimized CNN-RNN for Software Fault Prediction [J]. Int J Performability Eng, 2025, 21(1): 48-55.
[3]	Manu Banga. Enhancing Software Fault Prediction using Machine Learning [J]. Int J Performability Eng, 2024, 20(9): 529-540.
[4]	Aditya Dayal Tyagi, and Krishna Asawa. Influence Maximization in Social Network using Community Detection and Node Modularity [J]. Int J Performability Eng, 2024, 20(9): 552-562.
[5]	Ekta Singh, and Parma Nand. Efficient Multi-Class Facial Emotion Recognition using YOLOv9: A Deep Learning Approach for Real-Time Applications [J]. Int J Performability Eng, 2024, 20(9): 581-590.
[6]	Manpreet Singh, Gauri Jindal, Akshita Oberoi, and Rohan Dhangar. Improving Crime Detection Through Geo-MDA: A Hybrid Linear Regression Approach in Data Mining [J]. Int J Performability Eng, 2024, 20(8): 469-477.
[7]	Nilesh Shelke, Deepali Sale, Sagar Shinde, Atul Kathole, and Rachna Somkunwar. A Comprehensive Framework for Facial Emotion Detection using Deep Learning [J]. Int J Performability Eng, 2024, 20(8): 487-497.
[8]	Lakshya Vaswani, Sai Sri Harsha, Subham Jaiswal, and Aju D. Unravelling Complexity: Investigating the Effectiveness of SHAP Algorithm for Improving Explainability in Network Intrusion System Across Machine and Deep Learning Models [J]. Int J Performability Eng, 2024, 20(7): 421-431.
[9]	Rohit Kumar Verma and Sukhvir Singh. A Hybrid Framework of Resource Allocation using Firefly and Deep Learning in Big Data Scheduling [J]. Int J Performability Eng, 2024, 20(6): 333-343.
[10]	Priya Singh and Rajalakshmi Krishnamurthi. AgriGuard: IoT-Powered Real-Time Object Detection and Alert System for Intelligent Surveillance [J]. Int J Performability Eng, 2024, 20(4): 232-241.
[11]	Ujjwal Deep, Sushant Kumar, and Kanika Singla. Integrating Deep Learning Architectures for Enhanced Human Action Recognition: An Ensemble Approach [J]. Int J Performability Eng, 2024, 20(4): 253-262.
[12]	Atul Kumar and Gurpreet Singh Lehal. Layout Detection of Punjabi Newspapers using the YOLOv8 Model [J]. Int J Performability Eng, 2024, 20(3): 186-193.
[13]	Shou-Yu Lee, Yu-Sheng Chu, Tzu-Wei Hsu, I-Hsiang Yu, and W. Eric Wong. Enhanced Recognition Approach for Herb Medicine using YOLOv8 in Medical Information Systems [J]. Int J Performability Eng, 2024, 20(12): 713-722.
[14]	Meenakshi Chawla and Meenakshi Pareek. Hybridizing Intelligence: A Comparative Study of Machine Learning Algorithm and ANN-PSO Deep Learning Model for Software Effort Estimation [J]. Int J Performability Eng, 2024, 20(11): 668-675.
[15]	Rohit Chandra Joshi, Aayush Juyal, Abhijeet Mishra, Avni Verma, and Kanika Singla. Deep Learning-Based Face Emotion Recognition: A Comparative Study [J]. Int J Performability Eng, 2024, 20(1): 1-9.