LLMs at the Edge: Decentralized Power and Control

Most of the large language models' applications have been implemented in centralized cloud environments, raising concerns about latency, privacy, and energy consumption. This chapter examines the potential application of LLMs in decentralized edge computing, where computing tasks are distributed across interconnected devices rather than centralized hosts. Therefore, by applying approaches like quantization, model compression, distributed inference, and federated learning, LLMs solve the problems of limited computational and memory resources on edge devices, making them suitable for practical use in real-world settings.

Several advantages of decentralization are outlined in the chapter, such as increased privacy, user control, and enhanced system robustness. Additionally, it focuses on the potential of employing energy-efficient methods and dynamic power modes to enhance edge systems. The conclusion re-emphasizes that edge AI is the way forward as a responsible and performant solution for the future of decentralized AI technologies, which would be privacy-centric, high-performing, and put the user ...