Adaptive Edge–Cloud Collaboration Using Reinforcement Learning for Ultra-Low-Latency IoT Applications: A Comprehensive Review

Abstract

The exponential proliferation of Internet of Things (IoT) devices across mission-critical domains—such as autonomous vehicular networks, industrial automation, tele-surgery, and smart energy grids—has driven an unprecedented demand for real-time data processing with strict sub millisecond to low-millisecond latency bounds. Traditional cloud-centric compute paradigms fail to satisfy these stringent timing constraints due to wide-area network latency, dynamic jitter, and bandwidth bottlenecks. Conversely, edge computing brings computational capabilities closer to end devices but remains constrained by limited processing capacity, energy reserves, and transient operational workloads. To overcome these inherent trade-offs, adaptive Edge–Cloud Collaboration (ECC) frameworks have emerged, dynamically partitioning and offloading computational workloads across heterogeneous, multi-tier execution environments. However, traditional optimization techniques struggle to accommodate the highly dynamic, non stationary, and unpredictable nature of next-generation wireless IoT networks. Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) have rapidly gained prominence as powerful, model-free algorithmic paradigms capable of learning optimal offloading, resource allocation, and task scheduling policies in complex, stochastic edge-cloud environments. This paper presents a comprehensive, systematic review of state-of-the-art adaptive edge-cloud collaboration architectures driven by reinforcement learning for ultra-low-latency IoT applications. We establish a formal mathematical modeling framework based on Markov Decision Processes (MDP) and Markov Games, categorize contemporary deep RL methodologies (including Q-learning, DDPG, PPO, SAC, and Multi-Agent DRL), and evaluate offloading trade-offs across latency, energy consumption, bandwidth utilization, and service reliability. Furthermore, we provide detailed comparative analyses, highlight critical research gaps, present simulation based performance benchmarks, examine engineering applications, and chart future research directions toward federated learning integration, quantum assisted DRL, and 6G edge intelligence.

KEYWORDS: Edge–Cloud Collaboration, Ultra-Low Latency, Deep Reinforcement Learning, Internet of Things (IoT), Computation Offloading, Resource Orchestration, Mobile Edge Computing (MEC), Proximal Policy Optimization.