<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Diksha verma on Medium]]></title>
        <description><![CDATA[Stories by Diksha verma on Medium]]></description>
        <link>https://medium.com/@dikshavermaa8800?source=rss-5961aa2c6320------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*nxEw5z_gOE11I8XZ</url>
            <title>Stories by Diksha verma on Medium</title>
            <link>https://medium.com/@dikshavermaa8800?source=rss-5961aa2c6320------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:46:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dikshavermaa8800/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Twitter’s Recommendation Algorithm]]></title>
            <link>https://medium.com/@dikshavermaa8800/twitters-recommendation-algorithm-0bb80be2c45b?source=rss-5961aa2c6320------2</link>
            <guid isPermaLink="false">https://medium.com/p/0bb80be2c45b</guid>
            <dc:creator><![CDATA[Diksha verma]]></dc:creator>
            <pubDate>Sun, 29 Sep 2024 14:40:26 GMT</pubDate>
            <atom:updated>2024-09-29T14:40:26.243Z</atom:updated>
            <content:encoded><![CDATA[<p>Twitter is a platform designed to connect users with the most relevant, real-time information from around the globe. With millions of tweets posted daily, how does Twitter decide which ones to display in your timeline? This post breaks down how Twitter’s recommendation algorithm selects and ranks tweets to deliver a personalized experience.</p><h3>How Does Twitter Select Tweets?</h3><p>At the heart of Twitter’s recommendation system are various models and features that extract key insights from tweets, user profiles, and engagement data. These models aim to answer fundamental questions about Twitter’s network, such as predicting future interactions or identifying trending tweets within communities. By effectively addressing these queries, Twitter ensures more relevant recommendations for its users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GFarx8DIqy7h0gaD.png" /><figcaption>This diagram below illustrates the major components used to construct a timeline:</figcaption></figure><p>The recommendation pipeline consists of three main stages that utilize these core models and features:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*svwAVxtcHhJTDe1-.png" /></figure><p><strong>Candidate Sourcing</strong>: In this stage, the recommendation algorithm fetches the best Tweets from different sources to create a pool of candidate Tweets.</p><p><strong>Ranking</strong>: Once the candidate Tweets are collected, a machine learning model is used to rank each Tweet.</p><p><strong>Heuristics and Filters</strong>: After ranking, heuristics and filters are applied to further refine the recommendations. These heuristics and filters eliminate Tweets from users you’ve blocked, filter out NSFW (Not Safe for Work) content, and remove Tweets that you’ve already seen. .</p><p>The service that is responsible for constructing and serving the For You timeline is called Home Mixer. Home Mixer is built on Product Mixer, our custom Scala framework that facilitates building feeds of content. This service acts as the software backbone that connects different candidate sources, scoring functions, heuristics, and filters.</p><p>Let’s explore the key parts of this system, roughly in the order they’d be called during a single timeline request, starting with retrieving candidates from <strong>Candidate Sources</strong>.</p><h3>Candidate Sources</h3><p>Twitter employs several <strong>Candidate Sources</strong> to retrieve recent and relevant tweets for users. For every request, the system aims to extract the top 1,500 tweets from a vast pool of hundreds of millions. These tweets come from two main sources: <strong>In-Network</strong> (tweets from people you follow) and <strong>Out-of-Network</strong> (tweets from people you don’t follow). On average, the For You timeline is composed of 50% In-Network and 50% Out-of-Network tweets, although this ratio may vary depending on individual user behavior.</p><p>By embedding tweets into different communities, Twitter assesses the popularity of a tweet within a specific group. The more a tweet is liked by users in that community, the stronger its association with that community becomes.</p><h4>In-Network Source</h4><p>The <strong>In-Network</strong> source is the largest and most important candidate source. It focuses on providing relevant and recent tweets from users you follow. These tweets are ranked based on their relevance using a logistic regression model, and the top-scoring tweets are passed to the next stage of the recommendation pipeline.</p><p>A key factor in ranking In-Network tweets is <strong>Real Graph</strong>. This model predicts the likelihood of engagement between two users. The higher the Real Graph score between you and a tweet’s author, the more tweets from that person you will see in your timeline.</p><h4>Out-of-Network Sources</h4><p>For tweets from users you don’t follow, Finding relevant tweets from outside a user’s network poses a more complex challenge: How can Twitter determine if a tweet will interest you if you don’t follow the author? To solve this, Twitter uses two main approaches: <strong>Social Graph</strong> and <strong>Embedding Spaces</strong>.</p><h4>Social Graph</h4><p>The <strong>Social Graph</strong> approach estimates what you might find relevant by analyzing the interactions of the people you follow and those with similar interests. It predicts what you might find relevant by exploring the engagement activities of your network. For instance, it checks which tweets people you follow have liked or commented on and then ranks those tweets using a logistic regression model. This process answers key questions, such as:</p><ul><li>Which tweets have people I follow recently engaged with?</li><li>Who likes similar tweets to me, and what else have they liked recently?</li></ul><p>Based on the answers, Twitter generates a list of candidate tweets and ranks them using a logistic regression model. <strong>GraphJet</strong>, a real-time graph processing engine, is used to traverse the engagement and follow networks, answering these questions and powering 15% of the Home Timeline’s tweets. Though this approach is effective, <strong>Embedding Spaces</strong> have become the primary source of Out-of-Network recommendations.</p><h4>Embedding Spaces</h4><p>The <strong>Embedding Space </strong>are used to identify content similarity, this approach addresses a more general question: Which tweets and users are similar to my interests? Twitter generates embeddings, or numerical representations, for user interests and tweet content. These embeddings allow Twitter to measure similarities between users and tweets.These embeddings serve as a proxy for relevance in the timeline. These clusters are created through matrix factorization, enabling the algorithm to better understand community dynamics.</p><p>One of the most powerful embedding spaces on Twitter is <strong>SimClusters</strong>. Through <strong>SimClusters</strong> — groups formed around influential users — Twitter finds relevant content based on shared interests. These clusters are created through matrix factorization, enabling the algorithm to better understand community dynamics. Twitter currently tracks 145,000 communities, which are updated every three weeks. Users and tweets are represented within these communities, and both can belong to multiple communities simultaneously.</p><p>Some of the largest communities include:</p><ul><li><strong>News</strong>: A community focused on current events, political discourse, and global affairs.</li><li><strong>Pop Culture</strong>: Encompassing entertainment, celebrities, and viral internet trends.</li><li><strong>Tech and Startups</strong>: A hub for discussions around technology, innovation, and emerging startups.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nGxA-d2R04wAKL35.jpeg" /></figure><h3>Ranking</h3><p>The <strong>ranking</strong> stage is critical to determining the relevance and engagement potential of each tweet in the <strong>For You</strong> timeline. After collecting around 1,500 candidate tweets from various sources (In-Network and Out-of-Network), Twitter uses a machine learning model to score and prioritize these tweets. The goal is to show users the most engaging and relevant content from this pool.</p><h4>Key Aspects of the Ranking System:</h4><ol><li><strong>Neural Network Model</strong>: Twitter uses a neural network with approximately <strong>48 million parameters</strong> to rank tweets. This model is continuously trained using massive amounts of interaction data, which includes user behaviors such as Likes, Retweets, Replies, and other forms of engagement.</li><li><strong>Features</strong>: The model takes into account <strong>thousands of features</strong>, including:</li></ol><ul><li><strong>User preferences</strong>: Frequency of engagement with certain users, topics, and types of tweets.</li><li><strong>Content features</strong>: The nature of the tweet (text, images, videos, hashtags).</li><li><strong>Social connections</strong>: The strength of the relationship between the user and the tweet author.</li><li><strong>Contextual information</strong>: Time of day, current events, and trending topics.</li></ul><p>3.<strong> Scoring Process</strong>: The neural network outputs <strong>ten labels</strong>, each representing the probability of a specific engagement action (e.g., Like, Retweet, Reply). The model assigns a score to each candidate tweet based on these predictions. Tweets are then ranked from highest to lowest, according to their relevance scores.</p><p>4.<strong> Equal Treatment</strong>: At this point, all candidate tweets — whether from In-Network or Out-of-Network — are treated equally. The only factor that matters is the predicted relevance and engagement potential.</p><p>5.<strong> Continuous Learning</strong>: The model is updated frequently to ensure that it adapts to changing user behaviors and trends, improving its predictions over time.</p><h3>Heuristics, Filters, and Product Features</h3><p>After ranking, heuristics, filters, and various product features are applied to enhance the user experience. Some examples of these features include:</p><p><strong>Visibility Filtering:</strong> Filtering out Tweets based on their content and user preferences, including filtering Tweets from blocked or muted accounts.</p><p><strong>Author Diversity:</strong> Ensuring a balanced mix of Tweets from different authors to avoid an overwhelming number of consecutive Tweets from a single author.</p><p><strong>Content Balance:</strong> Striving to deliver a fair balance of in-network and out-of-network Tweets to provide a diverse feed.</p><p>Feedback-based Fatigue: Adjusting the score of certain Tweets based on negative feedback received.</p><p><strong>Social Proof:</strong> Including out-of-network Tweets in the feed only if they have a second-degree connection to the Tweet (e.g., someone you follow has engaged with the Tweet or follows the Tweet’s author).</p><p><strong>Conversations:</strong> Threading replies together with the original Tweet to provide more context.</p><p><strong>Edited Tweets:</strong> Detecting stale Tweets on a user’s device and replacing them with edited versions.</p><h3>Mixing Content: Creating a Seamless Timeline Experience</h3><p>The final step of Twitter’s recommendation process is <strong>content mixing</strong>, where tweets are blended with other non-tweet elements, such as advertisements, follow recommendations, and onboarding prompts. This comprehensive mix is then delivered to the user’s device for display. Twitter aims for a balance of various content types to maintain user engagement and satisfaction.</p><p>This recommendation system is remarkably efficient, processing about <strong>5 billion timeline requests per day</strong>, each completed in under <strong>1.5 seconds</strong>. This process uses<strong> 220 seconds of CPU time</strong>, making it roughly <strong>150 times</strong> faster than the latency perceived by users on the app.</p><p><strong>Reference</strong></p><p><a href="https://blog.x.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm">https://blog.x.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm</a></p><p><a href="https://github.com/twitter/the-algorithm">https://github.com/twitter/the-algorithm</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0bb80be2c45b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>