A Hybrid Approach to Fault Localization: Integrating LLMs with IR-Based Methods

Abstract

Fault localization aims to automatically localize buggy files, a key step in debugging tasks. Traditional Information-Retrieval-based fault localization (IRFL) methods often struggle due to the lexical gap between bug reports and source code. Inspired by the ability of Large Language Models (LLMs) to process both natural language and programming language, we propose a hybrid fault localization approach that integrates LLM-driven information extraction, semantic search, and relevance-matching techniques to improve fault localization accuracy. Our method utilizes LLMs to extract key information from bug reports, including keywords (variable names, function names, class names), error message verbatims, and technical descriptions. With the extracted keywords, we compute lexical similarity scores between bug reports and source code files and then rank the source code files accordingly. We also utilize text embedding models to encode the extracted bug reports and source code files and compute their semantic similarity to construct a ranked list of suspected buggy files. This semantic ranking is combined with lexical ranking based on the best-rank selection strategy to achieve the final list. Our approach is evaluated on six real-world Java projects from the Bench4BL dataset. Experimental results demonstrate that our approach outperforms the baseline methods by a substantial margin in terms of Top-K, Mean Reciprocal Rank (MRR), and Mean Average Precision (MAP) metrics.