SecurePrep: A Human-in-the-Loop Toolkit for Privacy-Preserving Prompt Engineering in the LLM Era

Abstract

The widespread adoption of large language models (LLMs) such as ChatGPT and Gemini has raised significant concerns regarding the inadvertent leakage of sensitive information through user prompts and document uploads. Existing privacy-preserving solutions for prompt engineering are often limited in their support for diverse document formats and lack meaningful human-in-the-loop (HITL) interaction. This paper introduces SecurePrep, an open-source toolkit that integrates human-computer interaction (HCI) and natural language processing (NLP) techniques to enable both end-users and developers to preprocess and sanitize prompts and documents before submission to LLMs. SecurePrep supports multiple document formats (PDF, DOCX, XLSX), provides an interactive redaction interface, and is designed for extensibility and integration into enterprise AI workflows. In verified benchmarks, SecurePrep achieves highest precision (0.467) with F1=0.306 and recall 0.242 for PII detection, preserves document structure, and blocks risky outputs. We present a comprehensive security analysis, novel algorithms for format-preserving redaction, and extensive evaluation across multiple dimensions including accuracy, scalability, and usability. Our results show that SecurePrep achieves an F1-score 2.5 times and precision 4.1 times that of a prominent baseline, while maintaining usability and regulatory compliance. We discuss the ethical implications for privacy-preserving AI and provide a roadmap for future research directions. SecurePrep is released as open-source software to promote safer AI adoption.