How large language models change the auto parallelized program translation landscape

Abstract

The transformer architecture has greatly improved the performance of large language models. Their natural language capabilities outperform previous iterations and have proven useful for translating natural language specifications into executable code. However, writing code for multicore and highly parallel systems using large language models remains difficult due to the complex nature of the task. Solutions based on large language models still compete with previous tried and true solutions such as auto-parallelizing compilers, dedicated programming languages, and even human experts. The dream of fully automating program parallelization remains out of reach, as even the most advanced models cannot overcome architectural limitations. Ultimately, the question shifts from whether or not using artificial intelligence to solve auto parallelized program translation tasks is worth it to the extent at which these solutions are useful before manual intervention is needed. Various risk factors are to be accounted for in making such a decision, and, while this thesis does not find a universally applicable answer, it provides a list of alternative solutions while acknowledging their shortcomings, leaving the decision to the project managers and product owners.