Malicious software, often known as malware, continues to pose a significant security concern to individuals, businesses, and governments in the modern digital age, especially with the exponential growth in malware attacks. Current malware detection systems rely on static and dynamic analysis of malware signatures and behaviour patterns to discover unknown infestations, but this process is inefficient and takes a lot of time. Thanks to evasive techniques like metamorphism and polymorphism, malicious software today may quickly alter its behaviour and generate a large amount of new malware. Most new viruses are variants of existing malware, hence it is useful that machine learning algorithms (MLAs) have recently been utilised to efficiently evaluate malware. For this, you'll need to put in a lot of time learning about and working with features. With advanced MLAs, such as deep learning, the feature engineering stage may be completely omitted. The algorithms' performance is biassed when trained on specific data, despite the fact that there have been some recent research in this field. We need to eliminate prejudice and conduct independent tests of these methods if we want to develop more effective methods of detecting zero-day malware. This work fills a gap in the literature by comparing traditional MLAs with deep learning architectures for malware detection, classification, and categorization utilising both public and private datasets. Both public and private datasets with timestamps are used for training and testing in the experimental study. Our novel approach to picture processing leverages optimal parameters in deep learning architectures and MLAs. An extensive experimental investigation has shown that deep learning architectures outperform conventional MLAs. Finally, our research concludes that a hybrid deep learning architecture that is both scalable and ready for real-time deployments may successfully detect malware visually. The hybrid approach using image processing, visualisation, and deep learning inside a big data framework is a novel and enhanced method for effectively detecting zero-day malware.

Malware detection, machine learning algorithms, deep learning, metamorphism, polymorphism, zero-day malware, image processing, big data framework, feature engineering.