This paper studies the protection principle of kernel hooks in the Windows operating system and proposes a deep-level kernel hook mining algorithm to solve the shortcomings of the interactive disassembler professional (IDA) cross-reference function. Firstly, the algorithm is used to dig out the internal calls of specified kernel functions and all the called positions of the kernel functions containing hooks. Then, we use Python to write mining algorithms based on the principle of function calls. Finally, we use C++ to write a driver program for passing-protection experiment. The performance of overprotection experiment is successful, which proves the effectiveness of the mining algorithm and the comprehensiveness of mining results.
LU Dengkai, YU Yongbin, YU Wenjian, TANG Qian, LIANG Shouyi
. Deep-Level Kernel Hook Mining Algorithm and Its Application in Software Security[J]. Journal of Applied Sciences, 2022
, 40(1)
: 61
-68
.
DOI: 10.3969/j.issn.0255-8297.2022.01.006
[1] Shen J, Cheng L, Fu X. Implementation of program behavior anomaly detection and protection using hook technology[C]//2009 WRI International Conference on Communications and Mobile Computing, IEEE, 2009, 3:338-342.
[2] Wang H D, Liao L. Research of process concealment based on technology of intercepting API calls[C]//20103rd International Conference on Computer Science and Information Technology, IEEE, 2010, 7:412-414.
[3] Yoshizaki K, Yamauchi T. Malware detection method focusing on anti-debugging functions[C]//2014 Second International Symposium on Computing and Networking, IEEE, 2014:563-566.
[4] Song Y, Shen Y, Zhang G. The new INLINE hook technology combination of hard-code technology and independent code injection[C]//20167th IEEE International Conference on Software Engineering and Service Science, 2016:521-525.
[5] Yousaf M S, Durad M H, Ismail M. Implementation of portable executable file analysis framework[C]//201916th International Bhurban Conference on Applied Sciences and Technology, IEEE, 2019:671-675.
[6] Yu C, Lai L. Research on model for verifying the integrity of software based on API hook[C]//201826th International Conference on Systems Engineering, IEEE, 2018:1-4.
[7] Af S M, Marhusin M F, Sulaiman R. Instrumenting API hooking for a realtime dynamic analysis[C]//2019 International Conference on Cybersecurity, IEEE, 2019:49-52.
[8] Grizzard J B, Levine J G, Owen H L. Re-establishing trust in compromised systems:recovering from Rootkits that Trojan the system call table[C]//European Symposium on Research in Computer Security. Berlin, Heidelberg:Springer, 2004:369-384.
[9] Wang Y, Gu D, Li W, et al. Virus analysis on IDT hooks of Rootkits Trojan[C]//2009 International Symposium on Information Engineering and Electronic Commerce, IEEE, 2009:224-228.
[10] Liu X, Liu R, Wu X. A secret inline hook technology[C]//20138th International Conference on Computer Science & Education, IEEE, 2013:913-916.
[11] Botacin M, De Geus P L, Grégio A. Leveraging branch traces to understand kernel internals from within[J]. Journal of Computer Virology and Hacking Techniques, 2020, 16(2):141-155.
[12] Zhang R, Wang L, Zhang S. Windows memory analysis based on KPCR[C]//2009 Fifth International Conference on Information Assurance and Security, IEEE, 2009, 2:677-680.
[13] Zhang C, Lin X, Lin S, et al. Study of handles mechanism in WRK[C]//2010 Second International Conference on Information Technology and Computer Science, IEEE, 2010:543-547.
[14] Javaheri D, Hosseinzadeh M. A framework for recognition and confronting of obfuscated malwares based on memory dumping and filter drivers[J]. Wireless Personal Communications, 2018, 98(1):119-137.
[15] Tsaur W J, Chen Y C. Exploring Rootkit detectors' vulnerabilities using a new windows hidden driver based Rootkit[C]//2010 IEEE Second International Conference on Social Computing, 2010:842-848.
