<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Zahra Zolfaghari on Medium]]></title>
        <description><![CDATA[Stories by Zahra Zolfaghari on Medium]]></description>
        <link>https://medium.com/@zahrazolfaghari00?source=rss-a71a00c18596------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*fjJle2NN9sxa3daW</url>
            <title>Stories by Zahra Zolfaghari on Medium</title>
            <link>https://medium.com/@zahrazolfaghari00?source=rss-a71a00c18596------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 23:35:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@zahrazolfaghari00/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ UUIDv7: The Fast, Unique, Ordered Identifier Every Scalable System Needs]]></title>
            <link>https://medium.com/@zahrazolfaghari00/uuidv7-the-fast-unique-ordered-identifier-every-scalable-system-needs-999e57eb0104?source=rss-a71a00c18596------2</link>
            <guid isPermaLink="false">https://medium.com/p/999e57eb0104</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Zahra Zolfaghari]]></dc:creator>
            <pubDate>Fri, 06 Jun 2025 13:05:50 GMT</pubDate>
            <atom:updated>2025-06-06T13:05:50.307Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HN6lK-uRhSN5RNP7ZJr1pg.png" /></figure><p>In scalable systems, one of the most underestimated decisions is choosing the right ID format. It seems like a small technical detail — until your database starts to choke under the weight of bad indexing and fragmented inserts.</p><p>In m<strong><em>y </em></strong><a href="https://medium.com/@zahrazolfaghari00/why-random-primary-keys-hurt-your-database-performance-understanding-clustered-vs-secondary-58a53a8cd5e7"><strong><em>previous article</em></strong></a>, I explained how random primary keys like UUIDv4 can silently degrade performance in relational databases. Now, there’s a new and better option on the scene:</p><blockquote><strong><em>Say hello to UUIDv7</em></strong><em> — a globally unique, time-ordered identifier designed for modern systems.</em></blockquote><h3>🔍 What You’ll Learn</h3><ul><li>What UUIDv7 is and why it matters</li><li>How it stacks up against UUIDv4, Auto-Increment, ULID, and Snowflake IDs</li><li>Why UUIDv7 helps improve insert and query performance</li><li>When (and how) you should use it in real projects</li></ul><h3>⏳ Traditional ID Strategies — And Why They Fall Short</h3><p>If you’ve built backend systems before, you’ve likely encountered these options:</p><p><strong>Auto-increment IDs</strong> are simple and fast, but they’re not globally unique — making them a poor fit for distributed systems.</p><p><strong>UUIDv4</strong> solves the uniqueness problem. It’s randomly generated and universally unique. But that randomness causes chaos in your database internals. Indexes get fragmented, inserts slow down, and range queries become inefficient.</p><p><strong>ULIDs</strong> and <strong>Snowflake IDs</strong> offer time-ordering and uniqueness — but often come with extra dependencies, ecosystem limitations, or inconsistent tooling support.</p><h3>💡 The Modern Need: Unique and Ordered IDs</h3><p>Today’s systems require identifiers that check all the boxes:</p><ul><li>Globally unique (for distributed systems and merging data safely)</li><li>Time-sortable (for efficient range queries and indexing)</li><li>Compatible with existing UUID-based tooling (like database types, logs, analytics)</li></ul><p>This is exactly where <strong>UUIDv7</strong> comes into play.</p><h3>🔬 What Is UUIDv7?</h3><p>UUIDv7 is part of the new UUID specification (RFC 9562), finalized in May 2023. It was created to bridge the gap between unordered UUIDs and performance-optimized identifiers.</p><p>It’s still a 128-bit UUID, just like UUIDv4, but with a smarter structure:</p><ul><li>The first part encodes the current Unix timestamp in milliseconds</li><li>The rest includes random bits for uniqueness</li><li>The version is set to 7, clearly distinguishing it</li></ul><p>This means UUIDv7 identifiers are <strong>lexicographically sortable by time</strong> while still being <strong>globally unique</strong>.</p><p>And because it uses the familiar UUID format, it plays nicely with existing systems, libraries, and databases.</p><h3>⚡ UUIDv7 in Action: Real Performance Gains</h3><p>I ran a test inserting 100,000 rows into MySQL using different ID strategies. Here’s what happened:</p><ul><li><strong>Auto-increment</strong> was lightning fast. As expected.</li><li><strong>UUIDv4</strong> was <em>painfully</em> slow. Inserts were scattered, and range queries crawled.</li><li><strong>ULID</strong> performed well — ordered and fast.</li><li><strong>UUIDv7</strong> matched ULID in performance, while staying within the UUID ecosystem.</li></ul><blockquote><em>The takeaway? UUIDv7 gives you </em><strong><em>the benefits of UUIDs without the fragmentation penalty</em></strong><em>.</em></blockquote><h3>🛠 When to Use UUIDv7</h3><p>Use UUIDv7 when your system needs:</p><ul><li>Unique IDs that can be generated independently across services</li><li>Time-based sorting and efficient range queries</li><li>High insert performance without B+ Tree fragmentation</li><li>Compatibility with UUID-based fields, types, and libraries</li></ul><p>You’ll especially love UUIDv7 in systems like:</p><ul><li>Event stores or activity feeds</li><li>Distributed databases</li><li>Logging and tracing systems</li><li>APIs that expose UUIDs externally</li></ul><h3>🚫 When Not to Use UUIDv7</h3><p>There are a few cases where UUIDv7 may not be ideal:</p><ul><li>You need <strong>short, human-readable IDs</strong> for URLs or public-facing tokens</li><li>You’re working in a very old or minimal environment without UUIDv7 support (though this is increasingly rare)</li></ul><h3>⚙️ How to Generate UUIDv7</h3><p>Good news: UUIDv7 support is growing fast across ecosystems.</p><h3>In Python</h3><pre>pip install uuid6</pre><pre>from uuid6 import uuid7<br>print(uuid7())</pre><h3>In Node.js</h3><pre>npm install uuid7</pre><pre>import { v7 as uuidv7 } from &#39;uuid7&#39;;<br>console.log(uuidv7());</pre><h3>In Go</h3><pre>import &quot;github.com/gofrs/uuid/v5&quot;</pre><pre>uuid.NewV7()</pre><h3>In Databases</h3><p>Most modern databases can store UUIDv7 using existing UUID or binary formats.</p><ul><li>In <strong>PostgreSQL</strong>, use the native uuid type.</li><li>In <strong>MySQL</strong>, store UUIDv7 as BINARY(16) and index it. Its timestamp-based structure makes clustered indexes significantly more efficient compared to UUIDv4.</li></ul><h3>🧠 Final Thoughts</h3><p>UUIDv7 is a near-perfect balance:</p><ul><li>As <strong>unique</strong> as UUIDv4</li><li>As <strong>ordered</strong> as auto-increment or ULID</li><li>As <strong>scalable</strong> as Snowflake IDs</li><li>As <strong>easy to adopt</strong> as any other UUID</li></ul><p>If you want IDs that are fast, future-proof, and frictionless, UUIDv7 is the upgrade you’ve been waiting for.</p><h3>📌 TL;DR</h3><ul><li>UUIDv4 = Safe, but slow and random</li><li>Auto-increment = Fast, but not globally unique</li><li>ULID/Snowflake = Solid choices, but less standard</li><li>UUIDv7 = ✅ Unique, ordered, fast, and widely compatible</li></ul><h3>🔗 Bonus: Dive Deeper</h3><p>Want to understand why UUIDv4 can be such a performance killer?</p><p>👉 <a href="https://medium.com/@zahrazolfaghari00/why-random-primary-keys-hurt-your-database-performance-understanding-clustered-vs-secondary-58a53a8cd5e7">Check out my earlier article: Why UUIDs Can Kill Your Database Performance</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=999e57eb0104" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Random Primary Keys Hurt Your Database Performance — Understanding Clustered vs Secondary…]]></title>
            <link>https://medium.com/@zahrazolfaghari00/why-random-primary-keys-hurt-your-database-performance-understanding-clustered-vs-secondary-58a53a8cd5e7?source=rss-a71a00c18596------2</link>
            <guid isPermaLink="false">https://medium.com/p/58a53a8cd5e7</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[index]]></category>
            <category><![CDATA[design]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Zahra Zolfaghari]]></dc:creator>
            <pubDate>Fri, 18 Apr 2025 12:29:39 GMT</pubDate>
            <atom:updated>2025-04-18T12:29:39.290Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gQnukl_dkB5trOFC1k-7YA.png" /></figure><h3>Why Random Primary Keys Hurt Your Database Performance — Understanding Clustered vs Secondary Indexes</h3><p><strong>Introduction</strong><br> When building applications, database performance is often treated as an afterthought — until things start slowing down. As your data grows, one of the most impactful decisions you make early on is how you design your primary keys.</p><p>Choosing the wrong type of primary key can quietly cripple your performance, especially when your dataset reaches hundreds of thousands or millions of rows.</p><p>In this article, I’ll walk you through:</p><ul><li>How databases organize your data using indexes.</li><li>Why random primary keys (like UUIDs) lead to performance headaches.</li><li>What you can do to avoid these common pitfalls.</li></ul><h3>🧠 Understanding How Databases Store Data</h3><p>Before we talk about primary keys, it’s important to understand how databases physically store data on disk.</p><p>Relational databases like <strong>MySQL (InnoDB)</strong> use a structure called a <strong>B+ Tree</strong> to store table rows. This is where indexes — and your primary key — become central.</p><p>At the heart of this storage model lies one key decision: <strong>which index controls the physical layout of your data?</strong></p><p>That’s the job of the <strong>Clustered Index</strong>.</p><h3>📊 Clustered Index: Your Table’s Backbone</h3><p>A <strong>Clustered Index</strong> determines the <strong>physical order</strong> of data in your table. Think of it like an already-sorted bookshelf, where each row is placed in order according to its primary key.</p><p>In MySQL’s InnoDB engine:</p><ul><li>The <strong>Primary Key</strong> is the <strong>Clustered Index</strong> by default.</li><li>Data rows are physically arranged on disk based on this key.</li><li>New records are inserted at the correct position to maintain this order.</li></ul><p>When your primary key follows a natural, sequential pattern (1, 2, 3...), inserts are efficient and range queries are blazing fast.</p><h3>🧭 Secondary Indexes: The Lookup Tables</h3><p>Beyond the primary key, you can create additional indexes, called <strong>Secondary Indexes</strong>, to speed up lookups on other columns.</p><p>However, secondary indexes don’t store the full row. Instead, they store:</p><ol><li>The indexed column(s).</li><li>A reference (pointer) to the <strong>Clustered Index</strong>.</li></ol><p>This means every query using a secondary index needs to:</p><ul><li>First use the secondary index to find matching keys.</li><li>Then use the clustered index to retrieve the actual row.</li></ul><p>In short:<br> <strong>Your table’s physical order still depends entirely on the primary key.</strong></p><h3>🤔 Why Primary Key Choice Matters</h3><p>So why is this background so important?</p><p>Because the moment you switch from <strong>sequential primary keys</strong> to <strong>random primary keys</strong> (like UUIDs), you create hidden problems that hurt both read and write performance.</p><p>Let’s break it down.</p><h3>🧨 The Problem with Random Primary Keys</h3><p>Using a <strong>random primary key</strong> (e.g. UUID) seems attractive for many reasons: they’re globally unique, hard to guess, and convenient for distributed systems.</p><p>But in a database, they create <strong>chaos</strong> in three major ways.</p><h3>1️⃣ Data Fragmentation</h3><p>With sequential primary keys, new data is appended at the end of the index — efficient and predictable.</p><p>With random keys, every new insert:</p><ul><li>Forces the database to search for the correct insertion point.</li><li>May trigger a <strong>page split</strong> (breaking one storage page into two).</li><li>Constantly reshuffles the B+ Tree.</li></ul><p>This causes severe <strong>fragmentation</strong>, increasing both disk usage and lookup time as your dataset grows.</p><h3>2️⃣ Cache Inefficiency</h3><p>Databases load and cache entire <strong>pages</strong> of data, not individual rows.</p><p>With sequential keys, related rows live close together on the same page, making repeated queries fast and cache-friendly.</p><p>With random keys, related data is scattered across many pages. Even simple queries can cause unnecessary page loads, leading to more <strong>cache misses</strong> and slower performance.</p><h3>3️⃣ Slow Range Queries</h3><p>If your primary key is sequential, queries like:</p><pre>SELECT * FROM orders WHERE id BETWEEN 1000 AND 2000;</pre><p>are lightning fast because the data is stored contiguously.</p><p>But with random keys, there’s no logical order, so range queries turn into inefficient, scattered disk lookups.</p><h3>🧪 Real-World Performance Test</h3><p>Let’s put this to the test.</p><p>A simple Python script was used to insert <strong>100,000 rows</strong> into a MySQL table. The only difference:<br> one version used <strong>Auto-Increment IDs</strong> (sequential), the other used <strong>UUIDs</strong> (random).</p><p>Primary Key Type Insert Time Auto-Increment (Sequential) <strong>0.017 seconds</strong> Random UUID <strong>1.29 seconds</strong></p><p>That’s an eye-popping <strong>75x slower</strong> just from the primary key strategy.</p><h3>💡 So, What Should You Do?</h3><p>Here are some practical guidelines:</p><p>✅ Use <strong>sequential IDs</strong> (Auto-Increment or timestamp-based) for your primary key.<br> ✅ If you need UUIDs for external references or uniqueness, store them in a <strong>secondary indexed</strong> column.<br> ✅ For distributed systems, consider alternatives like UUID_SHORT() (MySQL) or <strong>K-Sortable IDs</strong> (such as ULIDs, Snowflake IDs) which maintain both uniqueness and insertion order.</p><h3>🔥 Final Thoughts</h3><p>Your database’s performance is deeply tied to how it physically stores your data.<br> The primary key isn’t just a unique identifier — it shapes the entire structure of your table on disk.</p><p>Random primary keys:</p><ul><li>Fragment your data.</li><li>Worsen cache performance.</li><li>Kill range query efficiency.</li></ul><p>Being intentional about your primary key design is one of the simplest — yet most powerful — ways to build a fast and scalable system.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58a53a8cd5e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ MySQL Indexing Demystified: Best Practices, Pitfalls, and Performance Boosts]]></title>
            <link>https://medium.com/@zahrazolfaghari00/mysql-indexing-demystified-best-practices-pitfalls-and-performance-boosts-15838e7f7f6b?source=rss-a71a00c18596------2</link>
            <guid isPermaLink="false">https://medium.com/p/15838e7f7f6b</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[index]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Zahra Zolfaghari]]></dc:creator>
            <pubDate>Fri, 11 Apr 2025 10:20:32 GMT</pubDate>
            <atom:updated>2025-04-11T10:20:32.202Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*y94fBnNVHBslDxb80EFzMg.png" /></figure><h3>Introduction</h3><p>Indexes are one of the most powerful tools for optimizing database performance — yet they’re often misunderstood or misused.</p><p>In <strong>MySQL</strong>, proper indexing can turn a sluggish query into a lightning-fast operation. On the other hand, poor indexing can waste storage and even degrade performance.</p><p>In this guide, you’ll learn:</p><p>✅ How MySQL indexes work under the hood<br> ✅ Types of indexes and when to use them<br> ✅ Best practices and common pitfalls<br> ✅ Real-world examples with benchmarks</p><h3>1. What Are Indexes?</h3><p>An <strong>index</strong> is a data structure that allows MySQL to locate rows quickly without scanning the entire table. Think of it like a book’s index — instead of reading every page, you jump straight to the relevant section.</p><h3>🔍 How MySQL Uses Indexes</h3><ul><li>Faster lookups (WHERE clauses)</li><li>Optimized sorting (ORDER BY)</li><li>Efficient joins (JOIN operations)</li><li>Avoiding full table scans</li></ul><h3>🚀 Example: Without vs. With an Index</h3><p><strong>Without an Index (Full Table Scan):</strong></p><pre>SELECT * FROM users WHERE email = &#39;alice@example.com&#39;;</pre><ul><li>Performance: O(N) — checks every row.</li></ul><p><strong>With an Index (B-Tree Lookup):</strong></p><pre>CREATE INDEX idx_email ON users(email);<br>SELECT * FROM users WHERE email = &#39;alice@example.com&#39;;</pre><ul><li>Performance: O(log N) — uses a B-tree structure.</li></ul><h3>2. Types of MySQL Indexes</h3><h3>🔹 A. Primary Key Index</h3><ul><li>Uniquely identifies each row</li><li>Automatically clustered (data ordered by PK)</li><li>Best for: WHERE, JOIN, or ORDER BY usage</li></ul><pre>CREATE TABLE users (<br>  id INT AUTO_INCREMENT PRIMARY KEY,<br>  name VARCHAR(100),<br>  email VARCHAR(100)<br>);</pre><h3>🔹 B. Secondary Index (Single-Column)</h3><ul><li>Speeds up filtering on one column</li></ul><pre>CREATE INDEX idx_name ON users(name);</pre><h3>🔹 C. Composite Index (Multi-Column)</h3><ul><li>Indexes multiple columns (order matters!)</li><li>Follows the <strong>leftmost prefix rule</strong></li></ul><pre>CREATE INDEX idx_name_email ON users(name, email);</pre><p>✅ Works for:</p><pre>SELECT * FROM users WHERE name = &#39;Alice&#39;;<br>SELECT * FROM users WHERE name = &#39;Alice&#39; AND email = &#39;alice@example.com&#39;;</pre><p>❌ Doesn’t work efficiently for:</p><pre>SELECT * FROM users WHERE email = &#39;alice@example.com&#39;;</pre><h3>🔹 D. Unique Index</h3><ul><li>Ensures no duplicate values</li></ul><pre>CREATE UNIQUE INDEX idx_email ON users(email);</pre><h3>🔹 E. Full-Text Index</h3><ul><li>Optimizes full-text searches (vs. LIKE &#39;%term%&#39;)</li></ul><pre>CREATE FULLTEXT INDEX idx_content ON articles(content);<br>SELECT * FROM articles <br>WHERE MATCH(content) AGAINST(&#39;database optimization&#39;);</pre><h3>🔹 F. Covering Index</h3><ul><li>Contains all the columns a query needs</li><li>Avoids reading from the table (index-only scan)</li></ul><pre>CREATE INDEX idx_covering ON users(name, email);<br>SELECT name, email FROM users WHERE name = &#39;Alice&#39;;</pre><h3>3. How MySQL Chooses Indexes (The Query Optimizer)</h3><p>MySQL’s <strong>query optimizer</strong> decides whether to use an index based on:</p><ul><li><strong>Selectivity</strong> — how unique the values are</li><li><strong>Cardinality</strong> — number of distinct values</li><li><strong>Query complexity</strong></li></ul><h3>🧪 Check Index Usage</h3><p>Use EXPLAIN to see the execution plan:</p><pre>EXPLAIN SELECT * FROM users WHERE email = &#39;alice@example.com&#39;;</pre><ul><li>type: ALL → Full table scan ❌</li><li>type: ref → Index lookup ✅</li></ul><h3>4. When Indexes Don’t Help (or Hurt Performance)</h3><h3>❗ A. Over-Indexing</h3><ul><li>Slows down INSERT, UPDATE, DELETE</li><li>Wastes storage</li></ul><h3>❗ B. Wrong Index Type</h3><ul><li>Using a B-tree on low-selectivity columns (e.g., gender)</li><li>Misusing composite indexes</li></ul><h3>❗ C. Functions Prevent Index Usage</h3><pre>SELECT * FROM users WHERE YEAR(created_at) = 2023;<br>-- Won’t use index</pre><p>✅ Fix:</p><pre>SELECT * FROM users <br>WHERE created_at BETWEEN &#39;2023-01-01&#39; AND &#39;2023-12-31&#39;;</pre><h3>❗ D. OR Conditions</h3><pre>SELECT * FROM users <br>WHERE name = &#39;Alice&#39; OR email = &#39;alice@example.com&#39;;</pre><p>✅ Fix:</p><pre>SELECT * FROM users WHERE name = &#39;Alice&#39;<br>UNION<br>SELECT * FROM users WHERE email = &#39;alice@example.com&#39;;</pre><h3>5. Best Practices for Indexing</h3><p>✅ Index columns used in WHERE, JOIN, ORDER BY, GROUP BY<br> ✅ Use composite indexes wisely (follow leftmost prefix rule)<br> ✅ Avoid indexing low-selectivity columns (e.g., status, is_active)<br> ✅ Monitor unused indexes with: sys.schema_unused_indexes (MySQL 8+)<br> ✅ Rebuild fragmented indexes using: OPTIMIZE TABLE</p><h3>6. Benchmark: Index vs. No Index</h3><h3>🧪 Test Table (1M Rows)</h3><pre>CREATE TABLE orders (<br>  id INT PRIMARY KEY,<br>  user_id INT,<br>  amount DECIMAL(10,2),<br>  created_at DATETIME<br>);<br>-- Insert 1 million rows...</pre><h3>❌ Query Without Index</h3><pre>SELECT * FROM orders WHERE user_id = 500;<br>-- Execution time: ~500ms (full scan)</pre><h3>✅ Query With Index</h3><pre>CREATE INDEX idx_user_id ON orders(user_id);<br>SELECT * FROM orders WHERE user_id = 500;<br>-- Execution time: ~2ms (index seek)</pre><p><strong>Result:</strong> ~<strong>250x faster!</strong> ⚡</p><h3>Conclusion</h3><p>MySQL indexes are a <strong>double-edged sword</strong> — they can dramatically boost performance but also hurt it if misused.</p><h3>🔑 Key Takeaways</h3><ul><li>Use indexes on <strong>high-selectivity columns</strong></li><li>Follow the <strong>leftmost prefix rule</strong> with composite indexes</li><li>Avoid <strong>over-indexing</strong> — balance read/write performance</li><li>Always validate with EXPLAIN</li></ul><p>By mastering indexes, you’ll optimize your database to handle <strong>millions of rows</strong> with ease.</p><h3>💬 What’s your biggest indexing challenge?</h3><p>Let’s discuss in the comments — I’d love to hear your experience or help out! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15838e7f7f6b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advanced Python Performance Optimization Techniques]]></title>
            <link>https://medium.com/@zahrazolfaghari00/advanced-python-performance-optimization-techniques-639c1ab19464?source=rss-a71a00c18596------2</link>
            <guid isPermaLink="false">https://medium.com/p/639c1ab19464</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Zahra Zolfaghari]]></dc:creator>
            <pubDate>Tue, 01 Apr 2025 18:55:31 GMT</pubDate>
            <atom:updated>2025-04-01T18:55:31.253Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*KL-rVDyFat_ju7S7FdegmQ.png" /></figure><p>Python is celebrated for its readability and ease of use, but when it comes to performance, it can often lag behind compiled languages like C or Rust. However, with the right optimizations, you can significantly improve the performance of your Python code. In this article, we’ll explore several advanced techniques for optimizing Python performance, focusing on both CPU and memory efficiency.</p><h3>1. Use Built-in Functions &amp; Libraries</h3><p>Python’s built-in functions are written in C and are much faster than equivalent Python code.</p><h3>✅ Do This:</h3><pre>numbers = [1, 2, 3, 4, 5]<br>sum_result = sum(numbers)  # Faster than looping manually</pre><h3>❌ Avoid This:</h3><pre>total = 0<br>for num in numbers:<br>    total += num  # Slower</pre><h3>Why?</h3><ul><li>Built-ins like sum(), map(), filter(), and itertools are optimized at the C level, offering better performance over manual Python implementations.</li></ul><h3>2. Leverage Data Structures Efficiently</h3><p>Efficient data structures can significantly reduce the time complexity of common operations.</p><h3>a) Use set for Fast Lookups</h3><ul><li><strong>O(1)</strong> average time complexity for membership checks.</li><li>Ideal for removing duplicates or checking for existence.</li></ul><pre>unique_items = set([1, 2, 2, 3, 4])  # {1, 2, 3, 4}<br>if 3 in unique_items:  # Faster than checking in a list<br>    print(&quot;Found!&quot;)</pre><h3>b) Use collections.defaultdict &amp; Counter</h3><ul><li>Avoid manual checks for key existence.</li><li>Counter is optimized for counting elements.</li></ul><pre>from collections import defaultdict, Counter<br>word_counts = defaultdict(int)  # No KeyError<br>word_counts[&quot;hello&quot;] += 1</pre><pre>counts = Counter(&quot;hello world&quot;)  # Fast counting<br>print(counts)  # {&#39;h&#39;: 1, &#39;e&#39;: 1, &#39;l&#39;: 3, ...}</pre><h3>3. Optimize Loops with List Comprehensions &amp; Generators</h3><h3>a) List Comprehensions</h3><p>List comprehensions are typically faster than for loops because they are optimized in bytecode.</p><pre>squares = [x**2 for x in range(1000)]  # Faster</pre><h3>b) Generators (Memory Efficiency)</h3><p>Use yield for lazy evaluation, which prevents loading large datasets into memory.</p><pre>def generate_numbers(n):<br>    for i in range(n):<br>        yield i  # Memory-efficient</pre><pre>numbers = generate_numbers(1_000_000)  # Doesn&#39;t store all numbers at once</pre><h3>4. Memoization &amp; Caching</h3><h3>a) functools.lru_cache</h3><p>Memoization with lru_cache can significantly speed up recursive functions by caching previously computed results.</p><pre>from functools import lru_cache<br><br>@lru_cache(maxsize=128)  # Caches up to 128 calls<br>def fibonacci(n):<br>    if n &lt; 2:<br>        return n<br>    return fibonacci(n-1) + fibonacci(n-2)<br><br>print(fibonacci(100))  # Much faster due to caching</pre><h3>b) Manual Caching</h3><p>For custom caching logic, you can maintain your own cache dictionary.</p><pre>cache = {}<br>def expensive_function(x):<br>    if x not in cache:<br>        cache[x] = x ** 2  # Compute once, reuse later<br>    return cache[x]</pre><h3>5. Avoid Global Variables</h3><p>Accessing global variables is slower than accessing local ones, so keep variables local within functions whenever possible.</p><h3>✅ Do This:</h3><pre>def compute():<br>    local_var = 42  # Faster access<br>    return local_var * 2</pre><h3>❌ Avoid This:</h3><pre>GLOBAL_VAR = 42  # Slower access<br>def compute():<br>    return GLOBAL_VAR * 2</pre><h3>6. Use __slots__ to Reduce Memory Overhead</h3><p>By using __slots__, you can prevent dynamic attribute creation, which reduces memory usage.</p><pre>class Person:<br>    __slots__ = [&quot;name&quot;, &quot;age&quot;]  # Only allows these attributes<br>    def __init__(self, name, age):<br>        self.name = name<br>        self.age = age</pre><pre>p = Person(&quot;Alice&quot;, 30)<br># p.address = &quot;XYZ&quot;  # ❌ AttributeError (no dynamic attributes)</pre><h3>Why?</h3><ul><li>This optimization saves memory by avoiding the creation of __dict__, which is used to store instance attributes. This is particularly useful for applications involving millions of instances.</li></ul><h3>7. String Concatenation Optimization</h3><p>String concatenation using + in loops can be inefficient, as it creates temporary objects. Instead, use .join() for more efficient concatenation.</p><h3>✅ Fast:</h3><pre>words = [&quot;Hello&quot;, &quot;World&quot;]<br>sentence = &quot; &quot;.join(words)  # Efficient</pre><h3>❌ Slow:</h3><pre>sentence = &quot;&quot;<br>for word in words:<br>    sentence += word + &quot; &quot;  # Creates new strings each time</pre><h3>8. Profiling &amp; Identifying Bottlenecks</h3><p>Identifying bottlenecks is crucial before diving into optimization. Use Python’s built-in profiling tools to measure performance.</p><h3>a) timeit for Microbenchmarks</h3><pre>import timeit<br>time_taken = timeit.timeit(&#39;&quot;-&quot;.join(str(n) for n in range(100))&#39;, number=10_000)<br>print(time_taken)  # Compare different approaches</pre><h3>b) cProfile for Full Code Analysis</h3><pre>import cProfile<br>def slow_function():<br>    return sum(i**2 for i in range(1_000_000))<br>cProfile.run(&#39;slow_function()&#39;)  # Shows time per function call</pre><h3>9. Use NumPy &amp; Pandas for Numerical Work</h3><p>For numerical operations, <strong>NumPy</strong> and <strong>Pandas</strong> are far more efficient than Python’s native data structures.</p><h3>a) NumPy (Optimized Arrays)</h3><pre>import numpy as np<br>array = np.arange(1_000_000)  # Faster than Python list<br>sum_result = np.sum(array)  # Optimized C operations</pre><h3>10. Consider JIT Compilation (Numba) or Cython</h3><h3>a) Numba (Just-In-Time Compilation)</h3><p>Numba can speed up numerical functions by compiling them into machine code using the @jit decorator.</p><pre>from numba import jit<br><br>@jit(nopython=True)  # Compiles to machine code<br>def fast_sum(arr):<br>    total = 0<br>    for x in arr:<br>        total += x<br>    return total</pre><h3>b) Cython (Compile Python to C)</h3><p>Cython allows you to write Python-like code that is compiled into C extensions for faster execution.</p><pre># example.pyx<br>def cython_sum(long[:] arr):<br>    cdef long total = 0<br>    for x in arr:<br>        total += x<br>    return total</pre><h3>Final Thoughts</h3><p>Optimization in Python starts with <strong>measuring first</strong> (using cProfile, timeit), then targeting bottlenecks. Sometimes, improving the algorithm&#39;s time complexity (e.g., from <strong>O(n)</strong> to <strong>O(log n)</strong>) yields far better results than micro-optimizations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=639c1ab19464" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Django QuerySet Laziness: When Does Your Database Get Queried?]]></title>
            <link>https://medium.com/@zahrazolfaghari00/understanding-django-queryset-laziness-when-does-your-database-get-queried-2e9b69d15d88?source=rss-a71a00c18596------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e9b69d15d88</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[query]]></category>
            <dc:creator><![CDATA[Zahra Zolfaghari]]></dc:creator>
            <pubDate>Fri, 28 Mar 2025 17:07:10 GMT</pubDate>
            <atom:updated>2025-03-28T17:07:10.014Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*n7JxPol40i3_VeNIdcPskA.jpeg" /></figure><p>Django’s Object-Relational Mapper (ORM) is a powerful tool that allows developers to interact with databases without writing raw SQL. However, if you’ve ever noticed your Django code running slower than expected, the issue might stem from how QuerySets work.</p><p>At the heart of Django’s efficiency is <strong>laziness</strong> — a clever optimization that delays database queries until the last possible moment. Let’s dive into how this works and when your QuerySets actually hit the database.</p><h3>What Does “Lazy” Mean in Django QuerySets?</h3><p>A QuerySet is <strong>lazy</strong> because it doesn’t execute a database query immediately upon creation. Instead, it waits until you explicitly request the results. This allows you to chain multiple filters, exclusions, and other operations without triggering multiple database hits.</p><p>For example:</p><pre># No database query happens here!<br>queryset = Book.objects.filter(title__startswith=&quot;The&quot;).exclude(author=&quot;Smith&quot;)</pre><p>The above code constructs a query but doesn’t execute it. The database is only queried when the QuerySet is <strong>evaluated</strong>. Think of it like writing a recipe — you define the steps, but the cooking doesn’t start until you’re ready to serve.</p><h3>When Are QuerySets Evaluated?</h3><p>Evaluation happens when Django <em>materializes</em> the results. Here are common triggers:</p><h3>1. Iteration</h3><pre>for book in queryset:  # Hits the database here!<br>    print(book.title)</pre><h3>2. Slicing with a step</h3><pre>books = queryset[::2]  # Evaluates and loads all objects into memory!</pre><h3>3. Calling repr(), len(), or list()</h3><pre>print(queryset)  # Uses `repr()` and fetches the first 20 records<br>len(queryset)    # Executes SELECT COUNT(*)<br>list(queryset)   # Loads all records into a list</pre><h3>4. Boolean checks</h3><pre>if queryset:  # Executes SELECT … LIMIT 1<br>    print(&quot;Found results!&quot;)</pre><h3>5. Pickling or caching</h3><p>Storing a QuerySet in a cache or session triggers evaluation.</p><h3>Why Laziness Matters</h3><h3>1. Efficiency</h3><p>Avoid unnecessary database hits by chaining filters:</p><pre># Still one query, even with multiple filters!<br>books = Book.objects.filter(published=True).filter(rating__gte=4.5)</pre><h3>2. Reusability</h3><p>Modify QuerySets before evaluation:</p><pre>cheap_books = books.filter(price__lt=20)  # Extends the original query</pre><h3>3. Performance Optimization</h3><p>Only fetch data when necessary. For example, paginating results with [:10] adds LIMIT 10 to the SQL query instead of loading all records.</p><h3>Common Pitfalls</h3><h3>1. Accidental Evaluation</h3><p>Printing a QuerySet in debugging (e.g., print(queryset)) triggers a query. Use logging or debugging tools carefully.</p><h3>2. Reusing Evaluated QuerySets</h3><p>Once evaluated, a QuerySet caches its results. However, creating a new QuerySet re-queries the database:</p><pre>q1 = Book.objects.all()<br>list(q1)  # Evaluates the QuerySet<br>list(q1)  # Uses cache (no query)</pre><pre>q2 = Book.objects.all()<br>list(q2)  # New query!</pre><h3>3. Using len() Instead of count()</h3><p>len(queryset) loads all records into memory. Use queryset.count() for a lightweight SELECT COUNT(*).</p><h3>4. Checking Existence with bool() Instead of exists()</h3><p>Use exists() to optimize existence checks:</p><pre>if queryset.exists():  # Uses SELECT EXISTS(...) (faster)<br>    ...</pre><h3>Best Practices</h3><p>✔ <strong>Chain Filters</strong>: Combine operations to minimize database queries. ✔ <strong>Use </strong><strong>iterator() for Large Datasets</strong>: Prevent memory overload by streaming results. ✔ <strong>Leverage </strong><strong>select_related() and </strong><strong>prefetch_related()</strong>: Reduce query counts for related objects. ✔ <strong>Delay Evaluation</strong>: Build QuerySets first and evaluate them only when needed.</p><h3>Conclusion</h3><p>Django’s lazy QuerySets are a double-edged sword: they optimize performance but require awareness of when evaluation happens. By understanding triggers like iteration, slicing, and boolean checks, you can write faster, more efficient code.</p><p>Next time you build a QuerySet, ask yourself: <em>“Do I really need this data now?”</em> If not, let Django’s laziness work for you. 🚀</p><p><em>Found this helpful? Follow me for more Django tips!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e9b69d15d88" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>