Research

AI-Driven Test Case Generation: A Hybrid Approach Combining Rule-Based Systems with Large Language Models

This thesis investigates a hybrid approach to automated test case generation that combines deterministic rule-based scaffolding with Large Language Model correction. Evaluated on a production CAD application (55 user stories, 870 test cases), the system achieved 92% time reduction, $0.002 per test case, and 94.4% acceptance criteria coverage.

Key Findings

• •92% reduction in test creation time vs fully manual methods
• •First-pass quality rate of 72.9% (95% CI: 58.4%--84.3%)
• •Dominant failure mode: non-deterministic language in error-handling tests (38.9%)
• •Hybrid approach provides structural guarantees that pure LLM methods lack
• •LLM cost is negligible ($0.48 for 207 test cases) -- human review is the real cost

Research Questions

• 01RQ1: Does hybrid generation reduce time vs manual?
• 02RQ2: What is the first-pass structural quality rate?
• 03RQ3: What are the dominant failure modes?
• 04RQ4: How does hybrid compare to pure LLM approaches?