Building a Secret Scanner That Catches What Generic Rules Miss

Part 2 of my Application Security Fundamentals series.

The Gap in Most Secret Detection

Most secret scanning starts with known formats.

An AWS access key always starts with AKIA followed by 16 characters. A GitHub Personal Access Token starts with ghp_. These are gifts for anyone writing a scanner because they have well-defined formats with almost no ambiguity.

AKIA[0-9A-Z]{16}

That one line catches AWS access keys with almost no false positives. The challenge begins with everything that doesn’t have a vendor-issued format. An internal service token.

A config.password = "..." in a Python file.

A connection string with credentials embedded inside it.

Nobody publishes regexes for organization-specific credentials because those formats are unique to the environments where they’re created.

That got me wondering, purely as a personal exercise: how would I build a scanner capable of detecting those cases without generating an overwhelming number of false positives?

This article walks through the design decisions while exploring that question, and — more importantly — where the real engineering challenges showed up.

The Easy 80%: Known Formats

Known secret formats are the easiest wins.

Private keys, for example, are almost impossible to mistake for anything else.

-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----

If that string appears inside a commit, there is very little ambiguity.

The same applies to:

• Google API Keys (AIza followed by 35 characters)
• npm tokens (npm_ followed by 36 characters)
• GitHub Personal Access Tokens
• JWTs (three Base64URL-encoded segments separated by periods)

For these secrets, the pattern itself is the signal.

No entropy calculation is necessary. No additional filtering is necessary. A well-written regex already provides a high-confidence result.

Implementing these rules took relatively little effort and immediately covered a large portion of commonly leaked credential types.

The difficult part wasn’t detecting secrets with known formats. It was detecting the ones that don’t have one.

The Hard Part: Detecting keyword = value Without Drowning in Noise

A natural first attempt is to look for assignments such as:

(password|secret|api|[_-]?key)\s*[:=]\s*["']?(.{8,64})["']?

The exact implementation varies depending on the language or parser, but the overall idea stays the same.

Get Prianshu Bhaskar’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This correctly identifies something like:

apiKey = "xK9$$$$R4tY7z"

Unfortunately, it also matches things like:

cacheKey = "userProfile"
secretSauce = "none of your business"
password = "test123"

Technically, the regex is doing exactly what it was designed to do. The problem is that it treats everything equally.

A scanner that reports every one of these quickly becomes a scanner that people stop paying attention to.

At that point, the problem isn’t the regex anymore. It’s the lack of context.

Instead of making the regex smarter, I started treating it as only the first signal — everything else would be decided afterward.

Shannon Entropy: A Second Signal

One useful signal is Shannon entropy.

Think of entropy as a measurement of how unpredictable a string appears. Human-readable words contain predictable character patterns and generally produce lower entropy. Randomly generated credentials usually contain a wider variety of characters and therefore score much higher.

The entropy of a string is calculated using:

H = -Σ p(x) log2 p(x)

where p(x) represents the frequency of each character.

Fortunately, you don’t need to memorize the formula. The intuition is enough: strings with repeated characters score low, strings that appear random score higher.

password123 | ~3.0 | Predictable

aaaaaaaaaa | 0.0 | No randomness

For generic keyword-based detections, I calculate entropy only on the basic test value. The ideal threshold will vary depending on what’s being scanned and how much noise is tolerable.

That means something like:

cacheKey = "userProfile"

is ignored, while:

apiKey = "xK9$mQ2vL8pR4tY7z"

is much more likely to be reported.

Entropy, however, isn’t a verdict — it’s simply another signal. Short strings don’t contain enough information for entropy to be particularly reliable, and occasionally even fake placeholders can appear sufficiently random.

What I’d Do Differently Starting Over

If I were building this again, I’d begin with the known-format rules. They’re straightforward to implement and provide quick confirmation that the scanning pipeline is working correctly.

Only after that would I move to generic detection. Writing the regex is an afternoon’s work — reducing false positives is where most of the engineering effort goes.

I’d also spend much more time validating against open-source code — before trusting the results. Nothing teaches you about false positives faster than running a scanner against real code.

Finally, I’d treat placeholder lists and stopword lists as living documents. Every false positive teaches you something new, and those small improvements accumulate surprisingly quickly.

Final Thoughts

Before designing this, I assumed secret detection was mostly about writing enough regular expressions. It turned out the regexes were the easy part. The harder problem was deciding when not to report something.

Good secret detection isn’t built on a single clever heuristic. It’s built by combining several imperfect signals — known patterns, placeholders, entropy, and lightweight filtering — into a result accurate enough for someone actually to trust.

That’s ultimately what determines whether a scanner becomes part of a workflow, or just another tool whose warnings get ignored.

I write about Application Security and DevSecOps by exploring how security tools work under the hood, and by building small projects to understand their design more deeply.