Concurrency, Parallelism, and Memory Management in Programming Languages: Models and Trade-offs

Abstract

Modern software must exploit multiple processor cores, remain responsive,and manage memory correctly, and the way a programming languagesupports concurrency and manages memory is central to its design; this paper reviews the models and trade-offs. It distinguishes concurrency, thestructuring of a program as independently executing tasks, from parallelism,their simultaneous execution, and describes the principal models ofconcurrency: the sharing of memory among threads, which communicatethrough shared variables and require synchronisation, with the attendanthazards of data races and deadlock; the passing of messages between taskswith private state, as in the actor and communicating-sequential-processmodels, which avoid these hazards; and the asynchronous, event-driven model, which provides concurrency without many threads. It describes themanagement of memory: manual allocation and freeing, which give control at the risk of error; automatic reclamation through reference counting andtracing garbage collection, which give safety at some cost in overhead andpredictability; and ownership systems, which provide memory safety without garbage collection at compile time. It describes the interaction of concurrency and memory, through the memory model and the requirement of memory safety for safe concurrency. The study finds that these choices involve trade-offs among performance, safety, and ease of programming, so that languages adopt different combinations suited to their aims, and that recent designs seek to provide safety without sacrificing performance. It concludes that concurrency and memory management are central to language design. Certain figures are illustrative. KEYWORDS: Concurrency, Parallelism, Threads, Message Passing, MemoryManagement, Garbage Collection, Ownership, Data Races, ProgrammingLanguages