Transformer-Based Parallel Code Translation Systems

Abstract

Authors: Kameshwar Singh, Ajay Prasad

Abstract: The rapid growth of heterogeneous computing platforms has increased the demand for efficient parallel programming models across different hardware architectures. Translating code written for one parallel programming paradigm into another remains a challenging and time-consuming task, often requiring deep expertise in both source and target languages. Transformer-based neural networks, originally developed for natural language processing, have recently shown promising capabilities in modeling structured programming languages. This paper presents a comprehensive review of transformer-based parallel code translation systems, focusing on their architectures, training methodologies, datasets, and performance metrics. We analyze how attention mechanisms capture syntactic and semantic relationships in parallel code constructs such as threads, kernels, synchronization primitives, and memory hierarchies. The paper also discusses current limitations including scalability, correctness assurance, and hardware-specific optimizations. A comparative analysis with traditional compiler-based and rule-driven translation techniques is provided. Finally, future research directions are outlined to improve reliability, explainability, and deployment of transformer-based systems in real-world parallel computing environments.

Keywords: Parallel Programming, Code Translation, Transformers, CUDA, OpenMP, OpenCL, Attention Mechanism, Neural Code Models