How to Build Custom Data Detectors Without Regex: DLP for Context-Aware Detection

DLP systems have traditionally relied on regex pattern matching to identify sensitive information. While regex excels at finding patterns, it fundamentally can’t understand context. It’s a massive limitation that forces security teams into endless cycles of tuning expressions and triaging false positives.

Nightfall AI built prompt-based entity detection to solve this problem. You can see it in this short demo here:

We’ve also created a detailed walkthrough of how it works and why it matters for your security operations.

The Regex Problem

Pattern matching means when tracking prescription numbers, employee IDs, or any business-specific sensitive data, regex will match anything that fits your pattern, regardless of whether it's the sensitive data you care about. A purchase order number might have the same structure as a prescription number. A project code might look identical to an employee ID. Regex can't tell them apart.

This creates two operational challenges:

1. High false positive rates that waste analyst time
2. Constant tuning cycles as you try to exclude non-sensitive matches

How Prompt-Based Detection Works

Instead of relying solely on pattern matching, prompt-based detection uses an LLM to understand context alongside patterns. You describe what you're looking for, provide examples, and the system learns to distinguish between similar-looking but fundamentally different data types.

Let's walk through a real implementation.

Building a Prompt-Based Detector

The Use Case: A healthcare organization needs to track prescription numbers across their systems. These numbers follow a specific format that includes a four-digit year and must be from 2020 or later. The complication: their purchase order numbers sometimes match the same pattern.

Step 1: Define the Entity

Start by describing what you're detecting and its context. For prescription numbers, that means:

• Healthcare workflows: patient care, patient billing, and health insurance
• Pattern specifics: includes a four-digit year (2020 or later), case insensitive
• Associated keywords that typically appear near this data

Step 2: Provide Examples

This is where context-aware detection diverges from traditional regex. You supply both positive and negative examples:

• Positive examples: Actual prescription numbers in your expected format
• Negative examples: Data that matches the pattern but isn't what you're protecting (like those purchase order numbers)

The negative examples are critical. They teach the system what to ignore, eliminating false positives at the detector level rather than forcing you to handle them downstream.

Step 3: Validate Before Deployment

The system runs a confidence check: can it classify this data type with 75% or higher accuracy based on your description and examples? This hygiene check prevents you from deploying detectors that aren't ready, saving time and preventing alert fatigue.

In our example, the system confirmed readiness and correctly identified the purchase order pattern as a negative keyword: exactly what we needed it to learn.

Testing Against Real Data

We tested the detector against four files:

• A PDF with a prescription number
• A text file with a prescription number
• A purchase order document (the negative case)
• An Excel spreadsheet with a column of prescription numbers

After adding the detector to a Slack DLP policy, all four files were shared in a public channel. Within seconds, the system:

1. Correctly identified prescription numbers in the PDF, text file, and Excel spreadsheet
2. Scanned the purchase order but did not flag it as a violation—the negative example eliminated the false positive
3. Processed structured (Excel) and unstructured (PDF, text) data with equal accuracy

Why This Matters for Security Teams

Faster Time to Protection: Building this detector took minutes, not weeks of regex tuning. You define what matters to your organization in natural language, not brittle pattern syntax.

Reduced False Positives: Context awareness means the system understands the difference between prescription numbers and purchase orders, even when they follow similar patterns. This directly reduces analyst workload.

Business-Specific Coverage: Every organization has unique sensitive data—internal project codes, custom identifiers, proprietary formats. Prompt-based detection lets you protect these without becoming a regex expert.

Coverage Across File Types: The same detector works across PDFs, text files, spreadsheets, and messaging platforms. You define the entity once, and it's protected everywhere.

The Architecture Advantage

This approach represents a fundamental shift in how DLP systems identify sensitive data. Instead of asking security teams to translate business requirements into regex patterns, you describe what you need to protect. The system handles the complexity of distinguishing context from pattern.

For security teams managing dozens or hundreds of custom data types, this architectural change has compounding effects. Each detector you build becomes more precise with minimal effort. The time saved on tuning and false positive investigation accumulates across your entire detection library.

Ready to eliminate regex from your custom detection workflow? Prompt-based entity detection is available now in the Nightfall console. You can start building context-aware detectors for your organization's sensitive data today.

Get a personalized demo here: https://www.nightfall.ai/demo