Volume 19 No 11 (2021)
 Download PDF
Creating High-Performance Data Workflows with Hadoop Components
Sivananda Reddy Julakanti, Naga Satya Kiranmayee Sattiraju, Rajeswari Julakanti
Abstract
The growing demand for processing large volumes of data in real time has brought attention to Hadoop as a powerful framework for distributed data storage and analysis. The Hadoop ecosystem consists of a set of components that enable the creation of high-performance data workflows capable of handling Big Data challenges such as scalability, fault tolerance, and speed. This study explores the integration of key Hadoop components—such as Hadoop Distributed File System (HDFS), MapReduce, Apache Hive, Apache HBase, and YARN—to create efficient, scalable data workflows. By examining these components' individual and collective capabilities, we propose a methodology for building high-performance workflows that can support a variety of data processing needs. Through case studies and benchmarking tests, this research evaluates the performance gains achieved by optimizing these components for real-world use cases, focusing on speed, resource utilization, and fault tolerance. The results suggest that optimizing these components not only improves the efficiency of data processing tasks but also enhances scalability, making it possible to handle large and growing datasets effectively. The study concludes with practical recommendations for organizations looking to implement Hadoop-based data workflows that meet high-performance standards.
Keywords
Hadoop, High-Performance Data Workflow, MapReduce, HDFS, Data Processing Optimization
Copyright
Copyright © Neuroquantology

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Articles published in the Neuroquantology are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant IJECSE right of first publication under CC BY-NC-ND 4.0. Users have the right to read, download, copy, distribute, print, search, or link to the full texts of articles in this journal, and to use them for any other lawful purpose.