AI-assisted vulnerability explanations in CI pipelines

Abstract

Modern software development increasingly relies on continuous integration pipelines, where security issues need to be detected and understood early in the development process. Static analysis tools such as Semgrep can automatically identify potential vulnerabilities, but their findings may be difficult for developers to interpret and act on. This thesis examines whether large language model-generated plain-language explanations can improve the comprehensibility and actionability of Semgrep security findings in a CI pipeline context. To study this, a prototype pipeline was implemented to scan intentionally vulnerable code with Semgrep, select representative findings, and generate explanatory output using a large language model. The prototype was evaluated through a questionnaire study with software engineering students. The results suggest that AI-generated explanations improved participants’ ability to understand the meaning, security relevance, and possible remediation direction of the findings. However, the study also identifies reliability concerns, including the risk of incomplete, misleading, or overly confident explanations. The thesis concludes that AI-generated explanations can be useful as assistive support for interpreting static- analysis findings, but they should not be treated as authoritative security guidance.