OPTIMISING ENERGY EFFICIENCY IN CLOUD-BASED BIG DATA ENVIRONMENT USING LSTM-DWN REINFORCEMENT LEARNING

In the realm of cloud-based big data environments, the optimisation of energy efficiency stands as a critical challenge. This paper addresses this issue through the utilisation of LSTM-DWN (Long short-term memory with deep wavenet) reinforcement learning. We propose a novel approach that integrates deep learning techniques, specifically LSTM (Long short-term memory) networks and DWN (Deep WaveNet), with reinforcement learning principles to dynamically optimise energy consumption. The proposed method operates in a continuous feedback loop, where LSTM-DWN models are trained on historical data to predict future resource demands and energy consumption patterns. Reinforcement learning algorithms are then employed to adaptively adjust cloud configurations in real-time based on these predictions, aiming to minimise energy usage while maintaining service quality. The flow of the proposed method involves data pre-processing, LSTM-DWN model training, reinforcement learning agent implementation, and continuous optimisation of cloud resources. Experimental results demonstrate the efficacy of the approach, showcasing substantial energy savings compared to baseline methods. Specifically, our method achieves an average energy efficiency improvement of 65% across various workload scenarios. Furthermore, the proposed approach exhibits robustness in handling dynamic workloads and varying user demands, highlighting its potential for real-world deployment in cloud-based big data environments. This research contributes to the advancement of energy-efficient computing paradigms, paving the way for sustainable and scalable infrastructure cloud infrastructure for large data processing. © 2024, Scibulcom Ltd.. All rights reserved.