Jaccard Similarity Is Surprisingly Simple. Here’s Why It’s Useful
Have you ever compared two playlists and thought, “We actually listen to a lot of the same music”?
Or looked at two developers’ skills and noticed they know many of the same technologies?
That is essentially what Jaccard Similarity measures.
It answers one simple question:
How much do two sets have in common?
And the best part is that you don’t need advanced mathematics or machine learning to understand it.
Jaccard Similarity in one example
Imagine two developers.
Developer A: Python, FastAPI, PostgreSQL, Docker
Developer B: Python, Django, PostgreSQL, AWS
They have two skills in common:
Python, PostgreSQL
Across both developers, there are six unique skills:
Python, FastAPI, PostgreSQL, Docker, Django, AWS
So:
Jaccard Similarity = 2 / 6 = 0.33
Their skill-set overlap is roughly 33%.
That’s really the whole idea.
The formula
Jaccard Similarity is formally written as:
J(A, B) = |A ∩ B| / |A ∪ B|
Where:
Intersection (A ∩ B) = items both sets have in common
Union (A ∪ B) = all unique items across both sets
Or just remember:
Jaccard Similarity = Common items / Total unique items
Visual suggestion: Add a simple two-circle Venn diagram here. Label the circles “Developer A” and “Developer B”, with Python and PostgreSQL in the overlapping section. This would probably be the most useful visual in the entire article.
What does the score mean?
A Jaccard score always falls between 0 and 1.
0 means there is no overlap.
1 means both sets are identical.
For example:
A = {"Python", "Java"}
B = {"React", "Angular"}# Jaccard = 0But:
A = {"Python", "Java"}
B = {"Python", "Java"}# Jaccard = 1A score of 0.5 sits somewhere between those extremes, but that does not automatically mean “50% semantically similar”.
It means 50% of the combined unique items overlap.
There is also no universal threshold for deciding whether two things are similar. That depends on your data and use case.
What about Jaccard Distance?
Jaccard Distance simply looks at the same relationship from the opposite direction.
Jaccard Distance = 1 — Jaccard Similarity
So if:
Jaccard Similarity = 0.75
then:
Jaccard Distance = 0.25
Similarity asks, “How alike are they?”
Distance asks, “How different are they?”
That’s all there is to it.
Jaccard Similarity with text
Now things get more interesting.
Take these two sentences:
Text A: “Machine learning is useful”
Text B: “Machine learning is powerful”
Turn them into sets:
A = {"machine", "learning", "is", "useful"}B = {"machine", "learning", "is", "powerful"}They share:
machine, learning, is
That’s 3 common words.
There are 5 unique words in total.
So:
Jaccard Similarity = 3 / 5 = 0.6
The texts have a 60% word-set overlap.
Notice that I said word-set overlap, not 60% of the same meaning.
That difference matters.
Visual suggestion: A second small Venn diagram would work well here. Put useful on the left, powerful on the right, and machine, learning, is in the overlap.
Jaccard can compare words. It can’t understand them.
Consider:
“The customer purchased a car.”
and:
“The user bought a vehicle.”
A human immediately sees the relationship:
customer ≈ user
purchased ≈ bought
car ≈ vehicle
Basic Jaccard does not.
To Jaccard, car and vehicle are different tokens.
It measures lexical similarity, not true semantic similarity.
If your question is:
“Do these sentences mean the same thing?”
Jaccard probably isn’t enough.
But if your question is:
“How much vocabulary do these texts share?”
Get Pritam Guha’s stories in your inbox
Join Medium for free to get updates from this writer.
Jaccard can be a very good fit.
Implementing it in Python
Once you understand the formula, the implementation is almost boring. And that’s a compliment.
def jaccard_similarity(a, b):
set_a = set(a)
set_b = set(b)intersection = set_a & set_b
union = set_a | set_b return len(intersection) / len(union)
For example:
a = ["python", "fastapi", "postgresql", "docker"]
b = ["python", "django", "postgresql", "aws"]print(jaccard_similarity(a, b))Output:
0.3333Python finds the intersection, finds the union, and divides their sizes.
No model. No API call. No complicated processing.
Making it work with real text
For actual text, I needed a little preprocessing.
import reWORD_RE = re.compile(r"[a-z0-9']+")
def tokenize(text: str) -> frozenset[str]:
return frozenset(
WORD_RE.findall(text.lower())
)
def jaccard_similarity(
text_a: str | None,
text_b: str | None,
) -> float | None: if not text_a or not text_b:
return None tokens_a = tokenize(text_a)
tokens_b = tokenize(text_b) if not tokens_a or not tokens_b:
return None return len(tokens_a & tokens_b) / len(tokens_a | tokens_b)
Lowercasing ensures Machine and machine are treated as the same word. Tokenization removes unnecessary punctuation, and sets automatically remove duplicate words.
I used this approach when I needed a lightweight, deterministic way to measure lexical overlap between existing pieces of text.
The requirement wasn’t:
“Understand everything these texts mean.”
It was simply:
“How much do these texts overlap?”
For that problem, adding an embedding model would have been unnecessary complexity.
One small detail I found important
What should happen if one of the texts is missing?
Returning 0.0 might seem reasonable, but it actually means:
“I compared the texts and found no overlap.”
That isn’t what happened.
There wasn’t enough information to make the comparison.
So I return None.
Think of it this way.
I ask whether you and Rahul like the same movies.
You give me your movie list.
Rahul gives me nothing.
I can’t conclude that you have zero movies in common.
I simply don’t know.
That’s the difference between 0.0 and None.
There is one catch with common words
Consider:
“The man is walking to the car.”
“The woman is talking to the doctor.”
They describe different things, but they share words like:
the, is, to
Jaccard counts them.
One option is to remove common stopwords before calculating similarity.
Then the comparison becomes closer to:
{man, walking, car}
versus
{woman, talking, doctor}
Now the Jaccard score is 0, which makes much more sense.
But changing preprocessing also changes your scores. So if you use a similarity threshold, you should test and tune it again after changing the tokenizer.
Jaccard or Cosine Similarity?
I use this mental shortcut:
Jaccard Similarity
“How much do these sets overlap?”
TF-IDF + Cosine Similarity
“How similar are these documents when word frequency and importance matter?”
Embeddings + Cosine Similarity
“How similar are these texts in meaning?”
For:
{python, ai, backend}
and:
{python, ai, cloud}
Jaccard may be all you need.
For:
“The customer purchased a car.”
and:
“The user bought a vehicle.”
embeddings are probably a better choice.
The more sophisticated technique isn’t automatically the better one. It depends on the question you are trying to answer.
Where Jaccard is useful
Jaccard works particularly well when your data naturally behaves like sets.
You’ll see it used for things like recommendation signals, tags, document comparison, duplicate detection, search, customer behavior, and categorical data.
The common pattern is always the same:
How much do these two sets have in common?
If that’s your question, Jaccard is worth considering.
Final thoughts
If you remember only one line from this article, make it this:
Jaccard Similarity = Common items / Total unique items
It doesn’t understand language. It doesn’t know synonyms. It doesn’t care how impressive your vector database is.
It measures overlap.
And sometimes that is exactly what you need.
What I like about Jaccard is that it is simple, deterministic, inexpensive, and explainable. If someone asks why two things received a similarity score of 0.4, you can literally show them the intersection and union.
Sometimes the right solution isn’t the most sophisticated one.
