Perception-to-Control Pipelines in Autonomous Vehicles: A Review of Deep Learning Integration for Motion Control

Abstract

Autonomous driving has witnessed substantial advancements, yet achieving reliable and intelligent decision-making in diverse, real-world scenarios remains a significant challenge. The integration of deep learning into autonomous vehicle (AV) systems has fundamentally transformed how machines perceive, interpret, and respond to complex, dynamic environments. This review systematically examines perception-to-control pipelines—the end-to-end computational architectures through which sensor data is converted into real-time vehicle control commands—with particular attention to the role of deep neural networks at each stage of this pipeline. The paper surveys the evolution of modular and end-to-end architectures, analyses core perceptual subsystems including object detection, semantic segmentation, depth estimation, and sensor fusion, and evaluates deep learning-based approaches to path planning and motion control. The literature underpinning this review was identified through systematic searches of the following academic databases: Web of Science, Scopus, Google Scholar, and PubMed. Searches were conducted since January 2025 and encompassed publications from 1989 to 2026. Key methodologies, including convolutional neural networks, recurrent neural networks, transformer-based models, and reinforcement learning, are discussed in the context of their practical deployment in autonomous driving systems. The review also addresses the critical challenges of safety, robustness under distributional shift, interpretability, and real-time computational constraints. By synthesising findings from recent literature across robotics, computer vision, and control engineering, this article provides a comprehensive and structured overview of the current state of deep learning-driven perception-to-control pipelines, identifies prevailing gaps, and outlines directions for future research that will be essential in realising safe and scalable autonomous mobility.