Energy-aware Multi-agent RAG Planner for Edge Devices Using vLLM and Model Pruning

Abstract

It looks into addressing the growing need to minimize energy use in systems that apply retrieval with language generation. As the use of large language models (LLMs) increases, their energy and operational costs go up, so there is a real need to find energy-saving ways to work with them. Even with the development of RAG architectures, energy efficiency is frequently neglected, causing bigger computational requirements. To bridge this research gap, we expect an energy-aware RAG planning approach to make use of vLLM which is a highly effective and optimized language model serving system and also rely on model pruning strategies. Minimizing the energy used is the main goal, not deteriorating the accuracy and quality of the answers retrieved. Because our system uses lightweight serving from vLLM and removes unnecessary parameters in the language models, it finds a good balance between workload and quality of results. Among our findings are (1) a RAG planning algorithm that adjusts the necessary model complexity to match query requirements, (2) model pruning techniques developed for energy-saving purposes in RAG and (3) thorough testing that confirmed significant energy cost reductions—up to 40%—with similar prediction performance. This means there is a good chance making RAG models more sustainable could be possible by using both improved serving frameworks and reducing model size. This allows for AI applications to be built considering energy efficiency in any place where resources are scarce.