LLM-assisted unit test generation and verification for Python open-source projects

Abstract

This thesis evaluates the use of ChatGPT with the GPT-5.5 Thinking model for LLM-assisted unit test generation and code verification. The study investigates whether ChatGPT can generate executable pytest unit tests, whether the generated tests improve code coverage, and what errors appear in the generated tests. The experiment was conducted on three Python open-source projects: inflection 0.5.1, humanize 4.15.0, and validators 0.35.0.

For each selected function, ChatGPT was prompted once using a fixed prompt template. Existing project tests were not provided to the model during generation. The generated tests were recorded as raw tests and, when necessary, minimally fixed tests. Across the three projects, 105 raw LLM-generated test cases were produced, and 104 passed without modification. After one minimal correction, all generated tests passed. Coverage remained 99% in inflection and humanize, while validators improved from 89% to 90%. The only LLM-related error was an incorrect assertion.

The results suggest that LLM-generated tests can be useful for small Python utility functions, but human review remains necessary.