From RAG to Multi-Agent Systems: A Survey of Modern Approaches in LLM Development

The rapid evolution of intelligent chatbots has been largely driven by the advent of Large Language Models (LLMs), which have greatly enhanced natural language understanding and generation. However, the fast-paced advancements in generative Artificial Intelligence (AI) and LLM technologies present challenges for developers to stay up-to-date and to select optimal architectures or approaches from a wide range of available options. This survey article addresses these challenges by providing a overview of cutting-edge techniques and architectural application choices in modern generative chatbot development. We explore various approaches involving retrieval strategies, chunking methods, context management, embeddings, and the utilization of LLMs. Furthermore, we analyze paradigms such as naive Retrieval-Augmented Generation (RAG) compared to Graph-Based RAG, as well as single-agent versus multi-agent systems. We examine agent-based methodologies, comparing single-agent systems with multi-agent architectures, and analyze how multi-agent systems can proficiently handle intricate tasks, enhance scalability, and mitigate faults such as hallucinations through collaborative efforts. Additionally, we review tools and frameworks such as LangGraph that facilitate the implementation of stateful, multi-agent LLM applications. By categorizing and analyzing these modern techniques, this survey aims to present the current landscape and future directions in chatbot development.