An Enhanced Hybrid Model with Adaptive Learning for Robust Ransomware Detection: A Three Tier Stacked Ensemble Approach
DOI:
https://doi.org/10.70882/josrar.2026.v3i3.189Keywords:
Ransomware detection, Stacked ensemble, LSTM, XGBoost, Stochastic Gradient Descent, Adaptive learning, Cybersecurity, Machine learningAbstract
Ransomware attacks have become one of the most severe cybersecurity threats globally, demanding detection frameworks that can adapt to rapidly evolving attack vectors. This paper presents a three-tier stacked ensemble hybrid model integrating Long Short-Term Memory (LSTM) networks, Extreme Gradient Boosting (XGBoost), and Stochastic Gradient Descent (SGD) with an adaptive meta-learner for robust ransomware detection. Evaluated on the UGRansome dataset comprising 29,809 samples across benign, ransomware, and suspicious traffic classes the proposed framework achieves an overall accuracy of 99.40% and an F1-score of 0.99 across all classes. The adaptive learning component, implemented via SGD partial_fit updates, enables incremental retraining without full model reconstruction, ensuring resilience against concept drift. Comparative analysis against conventional baselines including Random Forest (97.2%), standalone XGBoost (98.1%), and SVM (96.5%), as well as recent hybrid approaches such as CNN–LSTM ensembles (98.3%) and RF–XGBoost combinations (98.7%), confirms that the proposed architecture delivers consistent improvements of 0.7–1.1 percentage points. Detection latency averaged 25–40 ms per sample, supporting real-time deployment. These results underscore the viability of adaptive ensemble architectures as scalable, self-learning defenses against modern ransomware.
References
Abbasi, S., Ali, A., & Khan, M. (2022). Early detection of ransomware using behavior-based classification with particle swarm optimization. Journal of Network and Computer Applications, 197, 103372. https://doi.org/10.1016/j.jnca.2022.103372
Almomani, I., Alkhayer, A., & El-Shafai, W. (2024). E2E-RDS: Efficient end-to-end ransomware detection system based on static-based ML and vision-based DL approaches. Sensors, 23(9), 4467. https://doi.org/10.3390/s23094467
Alpaydin, E. (2020). Introduction to machine learning (4th ed.). MIT Press.
Alraizza, A., & Algarni, A. (2023). Ransomware detection using machine learning: A survey. Big Data and Cognitive Computing, 7(3), 143. https://doi.org/10.3390/bdcc7030143
Abumed, H., Othman, M., & Hassan, R. (2023). Temporal behavioral analysis using LSTM for network intrusion detection. IEEE Transactions on Network and Service Management, 20(2), 1178–1192.
Bello, I., Chiroma, H., Abdullahi, U. A., Gital, A. Y., Jauro, F., Khan, A., Okesola, J. O., & Abdulhamid, S. M. (2021). Detecting ransomware attacks using intelligent algorithms: Recent development and next direction from deep learning and big data perspectives. Journal of Ambient Intelligence and Humanized Computing, 12, 8699–8717. https://doi.org/10.1007/s12652-020-02630-7
Bosansky, B., Hospodkova, L., Najman, M., Rigaki, M., Babayeva, E., & Lisy, V. (2024). Counteracting concept drift by learning with future malware predictions. arXiv preprint arXiv:2404.09352.
Brewer, R. (2021). Ransomware attacks: Detection, prevention and cure. Network Security, 2021(1), 5–9. https://doi.org/10.1016/S1353-4858(16)30086-1
Buczak, A. L., & Guven, E. (2016). A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys & Tutorials, 18(2), 1153–1176. https://doi.org/10.1109/COMST.2015.2494502
Chaganti, V., Srikanth, D., & Babu, A. (2023). Multi-view feature fusion for effective ransomware detection using deep learning. Journal of Information Security and Applications, 78, 103475. https://doi.org/10.1016/j.jisa.2023.103475
Cheng, J., & Zhang, G. (2024). Detecting stealthy ransomware in IPFS networks using machine learning. IEEE Access, 12, 45678–45692.
Connolly, L., & Wall, D. (2019). The rise of crypto-ransomware in a changing cybercrime landscape: Taxonomising countermeasures. Computers & Security, 87, 101568.
Cybersecurity Ventures. (2023). Global ransomware damage costs predicted to exceed $265 billion by 2031. Cybercrime Magazine. https://cybersecurityventures.com
Deng, Z., Zhang, L., & Liu, X. (2024). Deep reinforcement learning for ransomware early detection on portable executable headers. IEEE Transactions on Information Forensics and Security, 19(3), 1775–1786. https://doi.org/10.1109/TIFS.2024.3278904
Fernández, A., García, S., Galar, M., Prati, R. C., Krawczyk, B., & Herrera, F. (2018). Learning from imbalanced data sets. Springer.
Fernando, N. (2023). Static versus dynamic analysis in malware detection: A comparative review. Computers & Security, 130, 103234.
Fernando, N., & Komninos, N. (2024). FeSAD: Ransomware detection with adaptation to concept drift. Future Generation Computer Systems, 150, 456–467. https://doi.org/10.1016/j.future.2024.03.001
Gheibi, O., Weyns, D., & Quin, F. (2021). Applying machine learning in self-adaptive systems: A systematic literature review. ACM Transactions on Autonomous and Adaptive Systems, 15(3), 1–37.
Gulmez, B., Yildiz, M., & Aydin, M. (2024). XRan: An explainable ransomware detection framework using dynamic analysis. IEEE Access, 12, 34567–34579. https://doi.org/10.1109/ACCESS.2024.3290458
Hsu, C.-M., Yang, C.-C., Cheng, H.-H., Setiasabda, P. E., & Leu, J.-S. (2021). Enhancing file entropy analysis to improve machine learning detection rate of ransomware. IEEE Access, 9, 138345–138351. https://doi.org/10.1109/ACCESS.2021.3114148
Hwang, J., Kim, J., Lee, S., & Kim, K. (2020). Two-stage ransomware detection using dynamic analysis and machine learning techniques. Wireless Personal Communications, 112, 2597–2609. https://doi.org/10.1007/s11277-020-07166-9
Jegede, A., Fadele, A., Onoja, M., & Aimufua, G. (2022). Trends and future directions in automated ransomware detection. Journal of Computer and Social Informatics, 1(1), 17–41. https://doi.org/10.33736/jcsi.4932.2022
Jordaney, R., Sharad, K., Dash, S. K., Wang, Z., Papini, D., Nouretdinov, I., & Cavallaro, L. (2017). Transcend: Detecting concept drift in malware classification models. In Proceedings of the 26th USENIX Security Symposium (pp. 625–642). USENIX.
Khammas, B. M. (2020). Ransomware detection using random forest technique. ICT Express, 6(4), 325–331. https://doi.org/10.1016/j.icte.2020.11.001
Kokulu, F. B., Soneji, A., Bao, T., Shoshitaishvili, Y., Tong, M. Z. H., Doupé, A., & Ahn, G. J. (2019). Matched and mismatched SOCs: A qualitative study on security operations center issues. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (pp. 1955–1970). ACM.
Kumar, R., Zhang, X., Wang, W., Khan, R. U., Kumar, J., & Sharif, A. (2019). A multimodal malware detection technique for Android IoT devices using various features. IEEE Access, 7, 64411–64430.
Li, H., Wang, Z., & Liu, Y. (2022). Multimodal fusion for ransomware detection with weight self-learning. IEEE Transactions on Information Forensics and Security, 17(12), 4207–4219. https://doi.org/10.1109/TIFS.2022.3198372
López-Martín, M., Carro, B., Sánchez-Esguevillas, A., & Lloret, J. (2017). Network traffic classifier with convolutional and recurrent neural networks for Internet of Things. IEEE Access, 5, 18042–18050. https://doi.org/10.1109/ACCESS.2017.2742706
MacColl, J., Mackay, J., Sheridan, A., & Barker, J. (2024). Ransomware trends and detection methods: A 2023–2024 review. Journal of Cybersecurity, 10(1), tyae003.
Masum, M., Rahman, M. M., & Islam, R. (2022). Machine learning-based ransomware detection: A comprehensive study. Computers & Security, 122, 102883. https://doi.org/10.1016/j.cose.2022.102883
Mohurle, S., & Patil, M. (2017). A brief study of WannaCry threat: Ransomware attack 2017. International Journal of Advanced Research in Computer Science, 8(5), 1938–1940.
Moustafa, N., Turnbull, B., & Choo, K. K. R. (2019). An ensemble intrusion detection technique based on proposed statistical flow features for protecting network traffic of Internet of Things. IEEE Internet of Things Journal, 6(3), 4815–4830. https://doi.org/10.1109/JIOT.2018.2871719
Pendlebury, F., Pierazzi, F., Jordaney, R., Kinder, J., & Cavallaro, L. (2019). TESSERACT: Eliminating experimental bias in malware classification across space and time. In Proceedings of the 28th USENIX Security Symposium (pp. 729–746). USENIX.
Ryan, J. (2021). Ransomware and the problem of digital extortion. Crime, Law and Social Change, 76(3), 345–362.
Scaife, N., Carter, H., Traynor, P., & Butler, K. R. B. (2016). CryptoLock (and drop it): Stopping ransomware attacks on user data. In Proceedings of the 36th International Conference on Distributed Computing Systems (pp. 303–312). IEEE.
Sikder, A. K., Petracca, G., Aksu, H., Jaeger, T., & Uluagac, A. S. (2020). A survey on sensor-based threats and attacks to smart devices and applications. IEEE Communications Surveys & Tutorials, 22(2), 1125–1167.
Silva, J. A. H., & Hernández-Alvarez, M. (2021). Large scale ransomware detection by cognitive security. In 2021 IEEE Ecuador Technical Chapters Meeting (ETCM) (pp. 1–4). IEEE. https://doi.org/10.1109/ETCM53625.2021.9627642
Singh, A., Ikuesan, R. A., & Venter, H. (2022). Ransomware detection using process memory. arXiv preprint arXiv:2203.16871. https://arxiv.org/abs/2203.16871
Sophos. (2023). The state of ransomware 2023. Sophos Ltd. https://www.sophos.com/en-us/whitepaper/state-of-ransomware
Talabani, H. S., & Abdulhadi, H. M. T. (2022). Bitcoin ransomware detection employing rule-based algorithms. Scientific Journal of University of Zakho, 10(1), 5–10. https://doi.org/10.25271/sjuoz.2022.10.1.865
Tian, Y., Zhang, Y., & Zhang, H. (2023). Recent advances in stochastic gradient descent in deep learning. Mathematics, 11(3), 682.
UGR'16. (2016). UGR'16: A new dataset for the evaluation of flow-based intrusion detection. University of Granada. https://nesg.ugr.es/nesg-ugr16
Vinayakumar, R., Alazab, M., Soman, K. P., Poornachandran, P., Al-Nemrat, A., & Venkatraman, S. (2019). Deep learning approach for intelligent intrusion detection system. IEEE Access, 7, 41525–41550.
Wu, Y. C., & Chang, Y. L. (2024). Ransomware detection on Linux using machine learning with random forest algorithm. Authorea Preprints.
Zahoora, U., Khan, A., Rajarajan, M., Khan, S. H., Asam, M., & Jamal, T. (2022). Ransomware detection using deep learning based unsupervised feature extraction and a cost sensitive Pareto Ensemble classifier. Scientific Reports, 12, Article 15647. https://doi.org/10.1038/s41598-022-19443-7
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Anas Ibrahim, Darius Tienhua Chinyio (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- NonCommercial — You may not use the material for commercial purposes.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
