Tool support for semi-automated evaluation of software architecture by leveraging large language models

Abstract

Context. There is a rise in the recognition of sustainability as a key concern in software architecture. There are structured ways to evaluate this, such as the Software Architecture Assessment for Sustainability, but they require manual effort and are often time-consuming. Goal. The goal of this study is to reduce the time and manual effort required to evaluate the software architecture for sustainability by semi-automating steps that are time-consuming and require manual effort using Large Language Models (LLMs).

Method. We follow a design science research methodology to identify the requirements of the artifact and steps from the Software Architecture Assessment for Sustainability that can be benefited by automation, design, and develop the artifact that semi-automates the Software Architecture Assessment for Sustainability, and evaluate the artifact using quantitative metrics that calculate the semantic similarity between the output of the artifact and manual ground truth.

Results. We provide an artifact to semi-automate the Software Architecture Assessment for Sustainability.

Conclusions. Large Language Models can support the Software Architecture Assessment for Sustainability by providing a semi-automated tool.