Sets in Python :
Introduction
When working with data in Python, one of the most common challenges is dealing with duplicates. Whether you’re cleaning up a list of customer emails, analyzing survey responses, or processing log files, duplicates can skew results and waste resources. Python offers a powerful built-in data type called sets that makes handling uniqueness straightforward and efficient. Sets not only help remove duplicates but also provide fast operations for comparing and combining data.
In this article, we’ll explore how sets work, why they’re useful, and how you can apply them to real-world data-cleaning tasks. We’ll cover the basics of uniqueness, set operations, practical examples, and a real-world scenario to show their impact.
What Is a Set in Python?
A set in Python is an unordered collection of unique elements. Unlike lists or tuples, sets automatically eliminate duplicates. This makes them ideal for situations where uniqueness matters.
Key Characteristics:
- Unordered: Elements don’t have a fixed position.
- Unique: No duplicate values are allowed.
- Mutable: You can add or remove elements.
- Efficient: Membership tests (
in) are faster compared to lists.
Creating a Set
numbers = {1, 2, 3, 4, 4, 5} print(numbers)
Output:
{1, 2, 3, 4, 5}
- Notice how the duplicate
4was automatically removed. That’s the power of sets.
Why Sets Improve Performance
When checking if an element exists in a list, Python has to scan through each item until it finds a match. This is O(n) time complexity. Sets, however, use a hash-based structure, making membership checks average O(1). This means sets are much faster when dealing with large datasets.
For example:
emails = [“a@example.com”, “b@example.com”, “c@example.com”, “a@example.com”] # Convert to set to remove duplicates
unique_emails = set(emails) print(unique_emails)
Output:
{‘a@example.com’, ‘b@example.com’, ‘c@example.com’}
-Not only are duplicates removed, but checking if "b@example.com" exists in the set is significantly faster than checking in a list.
Common Set Operations
Sets come with powerful operations that make data comparison and cleaning easier.
1. Union (|) : Combines elements from two sets.
set1 = {1, 2, 3} set2 = {3, 4, 5} print(set1 | set2)
Output:
{1, 2, 3, 4, 5}
2. Intersection (&) : Finds common elements.
print(set1 & set2)
Output:
{3}
3. Difference (-) : Finds elements in one set but not the other.
print(set1 — set2)
Output:
{1, 2}
4. Symmetric Difference (^) : Finds elements that are in either set but not both.
print(set1 ^ set2)
Output:
{1, 2, 4, 5}
These operations are extremely useful when comparing datasets, such as finding overlapping customers or identifying missing entries.
Practical Examples :
Example 1: Removing Duplicates from a List
data = [10, 20, 20, 30, 40, 40, 50]
cleaned_data = list(set(data))
print(cleaned_data)
Get Sindhu Karnati’s stories in your inbox
Join Medium for free to get updates from this writer.
Output:
[10, 20, 30, 40, 50]
The order may change because sets are unordered, but duplicates are gone.
Example 2: Checking Membership Efficiently
names = {“Alice”, “Bob”, “Charlie”}
print(“Bob” in names)
print(“David” in names)
Output:
True
False
Membership checks are fast and efficient compared to lists.
Example 3: Finding Common Elements Between Two Lists
list1 = [“apple”, “banana”, “cherry”]
list2 = [“banana”, “cherry”, “date”]
common = set(list1) & set(list2)
print(common)
Output:
{‘banana’, ‘cherry’}
This is much simpler than writing nested loops to compare lists.
Real-World Use Case: Cleaning Customer Email Data
Imagine you’re working with a marketing team that has collected thousands of customer emails from different sources. Some emails are duplicated, and some lists overlap. Sending multiple emails to the same person wastes resources and annoys customers.
Here’s how sets can help:
list_a = [“alice@example.com”, “bob@example.com”, “carol@example.com”]
list_b = [“bob@example.com”, “dave@example.com”, “carol@example.com”]
# Remove duplicates
unique_a = set(list_a)
unique_b = set(list_b)
# Find all unique customers
all_customers = unique_a | unique_b
print(“All customers:”, all_customers)
# Find customers in both lists
common_customers = unique_a & unique_b
print(“Common customers:”, common_customers)
# Find customers only in list_a
exclusive_a = unique_a — unique_b
print(“Exclusive to list_a:”, exclusive_a)
Output:
All customers: {‘alice@example.com’, ‘bob@example.com’, ‘carol@example.com’, ‘dave@example.com’}
Common customers: {‘bob@example.com’, ‘carol@example.com’}
Exclusive to list_a: {‘alice@example.com’}
This approach ensures:
- No duplicates in the final list.
- Clear identification of overlapping customers.
- Efficient handling of large datasets.
Conclusion
Sets in Python are a beginner-friendly yet powerful tool for ensuring uniqueness and improving performance. They automatically remove duplicates, provide fast membership checks, and support intuitive operations like union, intersection, and difference. Whether you’re cleaning customer data, analyzing survey results, or comparing datasets, sets can save time and reduce complexity.
By incorporating sets into your workflow, you’ll not only write cleaner code but also handle data more efficiently. For anyone starting out in Python, mastering sets is a small step that delivers big benefits in real-world applications.
