Exploring and modelling human-AI collaboration effectiveness in software engineering using behavioural analysis and machine learning
Awalikara Galappaththige, Niranjith Kumara Premawansa (2026)
Lataukset:
Diplomityö
2026
School of Engineering Science, Laskennallinen tekniikka
Kaikki oikeudet pidätetään.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi-fe20260620100637
https://urn.fi/URN:NBN:fi-fe20260620100637
Tiivistelmä
The increasing adoption of Artificial Intelligence (AI) coding assistant tools such as ChatGPT, GitHub Copilot, Gemini and Cursor is transforming software development practices. This study investigates human–AI collaboration in software engineering by examining the relationship between collaboration behaviours, user perceptions and programming performance during AI-assisted coding tasks.
A quantitative task-based observational study was employed involving 92 participants, including software engineering students, computing undergraduates, interns and junior developers. Participants completed two AI-assisted Python programming tasks and behavioural, performance and perception-related data were collected. Behavioural indicators included prompt usage, prompt refinement, AI suggestion acceptance, testing behaviour, task completion activities and time investment. An automated code evaluation system was used to assess solution quality. Data were analysed using descriptive statistics, correlation analysis, regression techniques and clustering methods.
The findings indicate that effective human–AI collaboration extends beyond the use of AI-generated code. Several behavioural indicators were associated with programming outcomes, while trust, confidence and perceived usefulness showed positive relationships with collaboration effectiveness. The results highlight the importance of evaluating, verifying and refining AI-generated recommendations. This study provides empirical evidence on behavioural factors associated with AI-assisted programming performance and offers practical insights for educators, developers and organisations.
A quantitative task-based observational study was employed involving 92 participants, including software engineering students, computing undergraduates, interns and junior developers. Participants completed two AI-assisted Python programming tasks and behavioural, performance and perception-related data were collected. Behavioural indicators included prompt usage, prompt refinement, AI suggestion acceptance, testing behaviour, task completion activities and time investment. An automated code evaluation system was used to assess solution quality. Data were analysed using descriptive statistics, correlation analysis, regression techniques and clustering methods.
The findings indicate that effective human–AI collaboration extends beyond the use of AI-generated code. Several behavioural indicators were associated with programming outcomes, while trust, confidence and perceived usefulness showed positive relationships with collaboration effectiveness. The results highlight the importance of evaluating, verifying and refining AI-generated recommendations. This study provides empirical evidence on behavioural factors associated with AI-assisted programming performance and offers practical insights for educators, developers and organisations.