RuleAugment: A Hybrid Framework Combining Rule-Based Systems and Large Language Models for Natural Language to Visualization Tasks

Abstract

Data visualization plays an important role in conveying a large amount of information. Recent approaches utilized prompting techniques to ask large language models (LLMs) to respond codes that may not run correctly. To mitigate this problem, this paper presented RuleAugment, a hybrid framework combining a rule-based system and LLMs to simplify the task of converting natural language queries into visualization. RuleAugment handled query normalization and mapping, complexity classification, and Python code generation. The performance is evaluated on five datasets, focusing on query mapping accuracy, code generation accuracy, and graph quality. The framework achieves high query mapping accuracy (up to 98.5% with F1-Score 98.2%), accurate code generation (Exact Match Ratio of 94.5%), and high-quality graphs (average score of 4.8/5 for visual accuracy). While effective with simple data, RuleAugment faces challenges when handling complex, heterogeneous data sets that require improvements in query processing. RuleAugment shows great potential in automating data visualization, allowing users to focus on analysis rather than technical processing.