<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anjali Shrivastava on Medium]]></title>
        <description><![CDATA[Stories by Anjali Shrivastava on Medium]]></description>
        <link>https://medium.com/@vastava?source=rss-c1df9a412b37------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*T3pru4JHWeFuK49BRRs2bw.jpeg</url>
            <title>Stories by Anjali Shrivastava on Medium</title>
            <link>https://medium.com/@vastava?source=rss-c1df9a412b37------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 04:24:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vastava/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Annotating the web]]></title>
            <link>https://vastava.medium.com/annotating-the-web-b58aec70a807?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/b58aec70a807</guid>
            <category><![CDATA[twitter]]></category>
            <category><![CDATA[web]]></category>
            <category><![CDATA[mastodon]]></category>
            <category><![CDATA[web3]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Mon, 30 Dec 2024 15:47:53 GMT</pubDate>
            <atom:updated>2025-01-18T16:45:02.813Z</atom:updated>
            <content:encoded><![CDATA[<p>When I <a href="https://medium.com/@vastava/community-notes-and-the-future-of-journalism-1331b5eec395">wrote my piece about Community Notes</a>, I became interested in the idea of open web annotation. After a light research sesh, I found it was an idea that <strong>a lot</strong> of people had:</p><ul><li><a href="https://whymirror.github.io/">_why’s hoodwink.d</a> (2005–2009)</li><li><a href="https://en.wikipedia.org/wiki/Google_Sidewiki?useskin=vector">Google SideWiki</a> (2009–11)</li><li><a href="https://techcrunch.com/2008/10/08/reframe-it-retreads-web-annotation-as-a-browser-add-on/">Reframe It</a> (2008-present): still alive, but shifted focus to an enterprise polling solution.</li><li><a href="https://techcrunch.com/2011/10/31/hypothes-is/">hypothes.is</a> (2011-present): have found success by pivoting into the academic space.</li><li><a href="https://nymag.com/intelligencer/2014/12/genius-minus-the-rap.html">Genius.com</a> (2012-present): yes, this is the same lyrics site you’re thinking of.</li></ul><p>You’ll notice that many of these products spun up in the 2000s, a largely pre-social media time when the web was much more open. You’ll also notice that many of them failed out of the annotation space.</p><p>I think the reasons for failure are pretty obvious:</p><ol><li><strong>Scale</strong>: even then, the world wide web was large, and the number of pages with one or more public annotations on it would be a tiny, tiny fraction of the space. The only real way for this to work is if discoverability is baked into the platform (ie. a link aggregator like reddit for sites with annotations). And that would never happen because of…</li><li><strong>Walled gardens</strong>: Social media platforms have little incentive to link to third-party content that would take users outside their own ecosystem.</li></ol><p>But that doesn’t make the concept not worth exploring.</p><p><strong>Twitter’s missed opportunity</strong></p><p><a href="https://www.paulgraham.com/twitter.html">Many</a>, <a href="https://thenewstack.io/twitter-turmoil-we-need-an-open-protocol-for-public-discourse/">many</a> <a href="https://machined.substack.com/p/musk-should-make-twitter-a-protocol">people</a> have commented on the missed opportunity of Twitter operating as a protocol, including <a href="https://qz.com/jack-dorsey-said-making-twitter-a-company-was-its-orig-1849603325">Jack Dorsey himself</a>. Most of these arguments called for a decentralized platform where online discourse could take place, citing the importance of free/unmoderated speech.</p><p>To be honest, I never connected with this vision. And now that <a href="https://bsky.app/">bluesky</a> has been made into reality, I still don’t understand the value-add of Twitter being decentralized — I mean, <a href="https://joinmastodon.org/">mastodon</a> exists and it’s functionally the same. Sure, these platforms are decentralized but still very much contained.</p><p>Then I read about <a href="https://readwrite.com/what_twitter_annotations_mean/">Twitter Annotations</a>, an experimental feature that allowed developers to attach any metadata to tweets (eg. a tweet about a movie could have annotations for the movie’s title, rating, and showtimes). Effectively a global commenting system — and this, I can get behind.</p><p><strong>How could we implement this (in an ideal world)?</strong></p><p>In the spirit of <a href="https://www.robinsloan.com/lab/specifying-spring-83/">Spring </a>’83, I started to think about what I <em>want</em> from the Internet. If the constraints of existing platforms didn’t exist, what would be my ideal web annotation system?</p><p>What if users could “graffiti” web pages, like a live chat between the note-taker and anyone actively on that page? What if users could toggle between viewing the original page, and the annotated version?</p><p>I’m reminded of classroom overhead projectors, and have to acknowledge the nostalgia factor at play here. Still, it excites me. It’s collaborative, educational, contextual, and most importantly, accessible.</p><p>But as I say this, I know what I’m conceptualizing here would probably be too complex for my twitter-addicted brain. (follow me? <a href="https://x.com/anjali_shriva">https://x.com/anjali_shriva</a>)</p><p>I leave you with this: A true internet town square should have never been confined to one platform. Perhaps the true town square is the web itself. Or perhaps it is something that’s yet to be built.</p><p>(This call to action feels very flat. I hope I come back and edit this once I have something better.)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b58aec70a807" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I made an interactive map of the “TikTok mansion” craze]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://vastava.medium.com/i-made-an-interactive-map-of-the-tiktok-mansion-craze-87d5689ee41f?source=rss-c1df9a412b37------2"><img src="https://cdn-images-1.medium.com/max/1490/1*fbIOnF-AfA17_84h2GWI0w.png" width="1490"></a></p><p class="medium-feed-snippet">Amid pandemic, hundreds of teens move cross-country to form &#x201C;content houses&#x201D;</p><p class="medium-feed-link"><a href="https://vastava.medium.com/i-made-an-interactive-map-of-the-tiktok-mansion-craze-87d5689ee41f?source=rss-c1df9a412b37------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://vastava.medium.com/i-made-an-interactive-map-of-the-tiktok-mansion-craze-87d5689ee41f?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/87d5689ee41f</guid>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[content-strategy]]></category>
            <category><![CDATA[influencers]]></category>
            <category><![CDATA[tiktok-app]]></category>
            <category><![CDATA[tik-tok]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Thu, 19 Dec 2024 02:43:11 GMT</pubDate>
            <atom:updated>2024-12-19T02:43:11.054Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How and why I built an app to teach me makeup]]></title>
            <link>https://medium.com/swlh/how-and-why-i-built-an-app-to-teach-me-makeup-852ea46c8d72?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/852ea46c8d72</guid>
            <category><![CDATA[beauty-tips]]></category>
            <category><![CDATA[beautytech]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[makeup]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Fri, 13 Dec 2024 05:08:20 GMT</pubDate>
            <atom:updated>2024-12-13T05:08:37.739Z</atom:updated>
            <content:encoded><![CDATA[<h4>Overcoming Internet Noise to Forge a Personalized Approach to Beauty</h4><p>Beauty, to me, has always felt overwhelming. The sheer volume of makeup tutorials on YouTube and TikTok rarely felt relevant to me as a brown girl with cool-to-olive undertones. (obligatory appreciation for <a href="https://linktr.ee/glowbymonica">@makeupbymonicaa</a>, though who is always a brown girl slay!)</p><p>Searching for personalized beauty tools only led me to face shape analyzers that charged upfront fees and stopped at basic classifications, offering no real guidance. Frustrated (and, admittedly, having a huge chip on my shoulder), I set out to build my own.</p><h4>What I Built: <a href="https://stylesense.io/">StyleSense.io</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XiWECYuQVd1khYPTxbSjxQ.png" /><figcaption>Demo of face shape classification tool: <a href="https://stylesense.io/face_analyzer">https://stylesense.io/face_analyzer</a></figcaption></figure><p>I built a free tool that uses machine learning to identify facial features like prominent cheekbones, wide foreheads, rounded chins, etc. Each feature gets a confidence score — weak, moderate, or strong — based on prediction outputs from a series of regression models. From these scores, the model suggests up to two potential face shapes, because I recognize that most people don’t fit neatly into one category. My approach treats face shape identification as an art, not a science.</p><p>What sets stylesense.io and the models I’ve developed apart is its intellectual honesty. It doesn’t present rigid rules but instead offers general suggestions tailored to your unique features. And we take this idea further by developing <a href="https://stylesense.io/makeup_recommendations/face_contour">custom makeup guide filters</a>, which are effectively personalized tutorials tailored to your features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wDIyh-XEPQ0lSA9JXqaWFA.png" /><figcaption>Demo of face contour makeup guides: <a href="https://stylesense.io/makeup_recommendations/face_contour">https://stylesense.io/makeup_recommendations/face_contour</a></figcaption></figure><h3>The Response So Far</h3><p>Today, the app has 2,000 active users per day. And genuinely, hearing from users who find it helpful is why I continue to invest time and effort into it (this app barely breaks even, y’all).</p><p>When I first <a href="https://www.reddit.com/r/Makeup101/comments/140fwov/an_easy_way_to_identify_your_face_shape/">shared this tool on Reddit</a>, the feedback was incredible. The post became highly upvoted, with many people sharing how they felt similarly overwhelmed by traditional makeup advice. To this day, that Reddit post drives traffic to the app.</p><h3>Why It Matters</h3><p>This tool isn’t just about makeup; it’s about empowering users to embrace and exprss their sense of self. I feel that I’ve created something that is personal and genuinely helpful — a resource I wish I had when I was starting my own makeup journey. There’s still so much to do.</p><p>For now, I’m excited to keep improving and helping more people feel confident in their makeup journeys. In future installments, I’ll share more about how I’m approaching growth and monetization, and the plans I have to make the tool even better.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=852ea46c8d72" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/how-and-why-i-built-an-app-to-teach-me-makeup-852ea46c8d72">How and why I built an app to teach me makeup</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Community Notes, and the future of journalism]]></title>
            <link>https://vastava.medium.com/community-notes-and-the-future-of-journalism-1331b5eec395?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/1331b5eec395</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[twitter]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[fact-checking]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Thu, 07 Mar 2024 15:30:19 GMT</pubDate>
            <atom:updated>2024-03-07T15:31:00.126Z</atom:updated>
            <content:encoded><![CDATA[<h4>An exploration into making news more communal, and interactive</h4><p>I have been using Community Notes on Twitter/X for ~8 months, and I <strong>love</strong> it. I don’t use that word lightly — in a world of major platforms blatantly ripping each other off, Community Notes has emerged as a truly innovative feature. Of the more recent developments in social media, I find it the most valuable, interesting, and inspiring.</p><blockquote>Disclaimer: This is not an essay about Twitter/X. Nor is it an essay about Elon’s radical restructuring of the platform’s <a href="https://www.insider.com/elon-musk-laying-off-twitter-was-painful-bbc-spaces-interview-2023-4">staff</a>, <a href="https://www.theverge.com/2023/12/29/24018645/x-twitter-elon-musk-content-moderation-law-california-decision">content policies</a> or <a href="https://www.cnbc.com/2023/10/20/elon-musk-x-formerly-twitter-to-launch-two-new-subscription-tiers.html">strategy</a>. I will point out, though, that amid these contentious changes, the Community Notes feature seems to be universally beloved by the X user base (a group that is often biased towards outrage).</blockquote><p>Rather this is an essay about the implications of collaborative fact checking, and how consuming news can become more of an interactive, communal experience — walk with me.</p><p><strong>So, why does Community Notes work?</strong></p><p>For those who don’t know what I’m talking about, Community Notes is a feature on X that allows users to add contextual information to tweets (typically tweets with misinformation or ones that omit key details). It essentially is collaborative fact-checking. Here’s an example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PygcshvbjdBB9OZ-kUufLQ.png" /><figcaption>Courtesy of X’s <a href="https://communitynotes.x.com/guide/en/about/introduction">documentation</a> on community notes</figcaption></figure><p>It works remarkably well in correcting misinformation in an unbiased way. I recognize that individuals aren’t neutral, but groups of people approximate that ideal and the algorithm <a href="https://vitalik.eth.limo/general/2023/08/16/communitynotes.html">leverages</a> that.</p><p>Anyone can become a contributor as long as they verify their identity and have been an X user for &gt;= 6 months. As I said at the top, I joined back in July, and it’s been transformative for how I consume tweets. It’s made me a more intentional reader, and a more engaged one — I want to unpack why I think this is.</p><ol><li><strong>Firstly, Community Notes turns readers into active participants.</strong> It’s well <a href="https://www.nngroup.com/articles/participation-inequality/#:~:text=90%25%20of%20users%20are%20lurkers%20(i.e.%2C%20read%20or%20observe,other%20priorities%20dominate%20their%20time.">documented</a> that the vast majority of online users are lurkers (ie. they don’t contribute or post, but passively observe). I myself used to post quite often but started to drop off once my social circle left the platform. In any case, there’s a mental block that most of us lurkers cannot overcome in order to post (namely, getting past the anxiety of “Why would anyone care?”). By virtue of being anonymous, Community Notes has a lower barrier to entry than explicitly tweeting. And because its main purpose is to target misinformation, no one will be left wondering why anyone would care.</li><li><strong>Secondly, active participation makes me an empowered stakeholder.</strong> I now feel a sense of ownership, and even responsibility, for the discourse ecosystem on Twitter, however slight. And I have some agency to shape the discourse and ensure that good-faith discussion is prioritized.</li><li><strong>Thirdly, it (implicitly) teaches media literacy and critical thinking.</strong> After contributing to Community Notes, I’ve become better at discerning reliable information from misinformation on Twitter. I believe the recommendation algorithm has gotten better too, although there’s no way to quantify that. Whether someone is enrolled in Community Notes or not, its very presence encourages a more intentional reading approach. Users are aware that they need to critically evaluate the information in their feed, especially if it has a note appended to it.</li></ol><p>The intention of Community Notes is not always to correct the offending tweet, but to add contextual information. I like to think of them as annotations — and they are an effective reminder that multiple perspectives exist.</p><p><strong>And why should newsrooms care?</strong></p><p>Yes, newsrooms have in-house fact-checking resources. There’s no immediate need to use the public for fact-checking in this way. They also would never deal with misinformation at the same scale that a platform like Twitter would, because journalists are generally committed to the truth.</p><p>I actually think for a private newspaper, we shouldn’t allow random people on the internet to challenge journalists’ authority in an unthoughtful way (see: <a href="https://vastava.medium.com/media-bias-charts-oversimplify-a-complex-non-issue-7fdddd78a4f1">my rant about media bias charts</a>).<strong> </strong>But<strong>,</strong> I have to acknowledge that newsrooms have an image problem, and I believe that letting readers into the process may help to win back their trust.</p><p>As I see it, there’s two ways to “let readers in”:</p><ol><li>Transparency: I have faith in journalists because at one point, I worked in a newsroom. I’m intimately familiar with the process — this is <a href="https://www.niemanlab.org/2023/12/to-build-trust-news-outlets-prioritize-transparency/">not a new idea</a>, and I’m not going to expand on it much in this essay.</li><li>Turning readers into active participants, similar to Community Notes. Newsrooms do a little bit of this in allowing readers to leave comments on some articles, but there’s room for further experimentation.</li></ol><p>Community Notes typically target biased language that journalists by training do not engage in. So you may not see the point — but as I’ve hinted at, there’s another benefit.</p><p><strong>The truth is that Community Notes are an engagement hack.</strong></p><p>Let’s be frank — people don’t read news as much. But they consume it indirectly, via instagram graphics, commentary youtubers, reddit megathreads, discussion boards and of course, tweets.</p><p>My personal theory is that we like reading with a “take” — news packaged in this way gives readers the relevant information, but also the manner in which they can naturally weave it into a conversation. I’m guilty of doing this, a lot.</p><p>Presenting commentary alongside reporting could be another way to hook readers into <strong>actually reading</strong> the story. I get why newspapers would not want to bias their reporting in this way — and opinion sections exist for a reason. So why not lean on existing readership to provide commentary?</p><p>This may be a foreign concept for anyone who didn’t grow up on social media, but the age of passively consuming content is over — for better or worse, we are all able to create content now, and most of us want to play a role in how information is communicated and shared, even if we don’t directly create it.</p><p>Line-by-line comments are not new; both Medium and Wattpad allow them (and probably other products I’m not aware of). Whenever I see a piece on Medium with several inline comments, I get excited — it simulates having a dialog with the author, and is much more dynamic than reading the story in a vacuum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*M298wVP40T_GdWo1JaLYIA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*zQB2HGqViE-tOAeGu1d4xA.png" /><figcaption>Screenshots of Medium and Wattpad comments interface.</figcaption></figure><p>And although it’s been a long time since I actively read Wattpad, I remember feeling the same about the comments there. Especially as a teenaged girl, I felt some residual shame about reading amateur stories on Wattpad, and the <em>thousands</em> of comments showed me I wasn’t alone in that enjoyment. It was <a href="https://blog.hubspot.com/marketing/social-proof-examples">social proof</a>, and it kept me on the platform.</p><p>I understand that newsrooms might be reluctant to adopt this model, given that user-generated content (UGC) arguably <a href="https://www.huffpost.com/entry/audience-atomization-over_b_157807">eroded</a> their authority in the first place. Newsrooms are slow to change policies by design; there are certain trade-offs journalists weigh that the “move fast, break things” crowd will not consider.</p><p>But that doesn’t mean they should remain static when every other major source of content evolves by the quarter. I believe there are ways to adapt to the current media ecosystem without compromising on principles.</p><p><strong>What are the principles of community notes that could apply to news?</strong></p><ol><li><strong>Behind-the-scenes &amp; transparency: </strong>When reading a news story, it’s difficult to feel the presence of authors, peers, readers — that’s why I mainly consume news on reddit and other forums. It extends the canvas of the story. Meta commentary is powerful, it builds trust between readers, in the institution and process and builds commentary.</li><li><strong>Interactivity: </strong>Hyperlinks and embeds have already been embraced, but we can do more. Longform data visualization stories is one example that has clearly served newsrooms well, but they’re not easily shareable to other platforms</li><li><strong>Social Proof: </strong>Reading news is an isolating experience, especially if the content is demoralizing — a sense of community (with readers, or authors) can help restore hope. This could indirectly address another complaint, in that only editors are the sole arbiters of what’s “newsworthy” — the community of readers can platform certain stories, by discussing and sharing it.</li></ol><p><strong>And what are some guardrails to keep in mind?</strong></p><ol><li>Should not sacrifice original reporting, quality or credibility.</li><li>Should not throw the journalist under the bus</li><li>Should not assume infallibility, either of journalists or readership</li></ol><p><strong>Adhering to these principles can build out a whole ecosystem of tools.</strong></p><p>Below are some mocks of example experiences that newsrooms could implement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v2FFL2KdFcuLt2cAo43RyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*QSx1J2BP8X5SAyBdJau2oA.png" /><figcaption>Mocks of Line-by-Line comments (left) and inline corrections (right) in a news article</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*Gp5BearKswiijtJLogKBJQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*d4rU9kA2S8BnQqLYb-Fy1Q.png" /><figcaption>Mocks of Readership Networks (left) and Real-Time Metrics (right) appearing alongside a news article</figcaption></figure><p>And these are just surface-level ideas — newsrooms could experiment with <a href="https://vastava.medium.com/stop-using-machine-learning-to-measure-media-bias-adaed698f955">machine learning models</a> to generate live summaries, or borrow from the Wikipedia model to host community “sparknotes”.</p><p><strong>In conclusion…</strong></p><p>I want to acknowledge that there’s real risk to the ideas I’m proposing. I know that moderating comments is a thankless, stressful job; I know that comments could become another avenue for nameless users to funnel their rage into; I know that online users behave differently when they feel “perceived,” and that there’s appetite to <a href="https://twitter.com/julesterpak/status/1765583784307462526">hide public-facing metrics</a>; and I know that journalists already deal with an unreasonable amount of hate and toxicity.</p><p>But, my trade is data science and I believe in the spirit of experimentation. I also believe in the wisdom of crowds. Wikipedia is a modern miracle — there are so many reasons it shouldn’t work, and yet. Tech companies have pioneered using UGC to engage and bait free audiences in order to strengthen network effects; news has barely survived this upheaval of the digital ecosystem, and the way to regain ground may lie in the tactics that once posed an existential risk.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1331b5eec395" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stop using “machine learning” to measure “media bias”]]></title>
            <link>https://vastava.medium.com/stop-using-machine-learning-to-measure-media-bias-adaed698f955?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/adaed698f955</guid>
            <category><![CDATA[sentiment-analysis]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[media-bias]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Wed, 07 Feb 2024 20:16:02 GMT</pubDate>
            <atom:updated>2024-02-07T20:16:02.324Z</atom:updated>
            <content:encoded><![CDATA[<p>Media bias discourse is both worn-out and doesn’t go far enough. The landscape is overrun with “neutral” analysts using some kind of sentiment analysis to lazily label something as right or left. This <a href="https://arxiv.org/pdf/2109.00024.pdf">MIT paper</a> that used “machine learning” to assess “media bias” was lauded as a <a href="https://www.unite.ai/mit-measuring-media-bias-in-major-news-outlets-with-machine-learning/">data-driven study</a> that objectively showed bias in reporting. This has happened <a href="https://www.rochester.edu/newscenter/study-of-headlines-shows-media-bias-growing-563502/">over</a>, and <a href="https://ground.news/about">over</a>, and <a href="https://www.pnas.org/doi/10.1073/pnas.2202197119">over</a> again. And this approach rarely passes the sniff test for me.</p><p>The researchers in the MIT paper scored publishers based on how frequently they used politically charged language (e.g. <em>“undocumented immigrant”</em> vs. <em>“illegal immigrant”</em>) — blindly searching for “biased” phrases ignores the fact that journalists employ rhetorical devices like sarcasm, <em>quotes</em> and nuance.</p><p>What is the goal here? If the problem of media bias is solely due to specific phrases, then I have a foolproof solution for ending it: CTRL+F and replace.</p><p>My point is that language does not exist in a vacuum. Telling a dark joke among friends is perfectly acceptable. To your boss… not so much. You have to consider the context.</p><p>So if I had to measure the media landscape, how would I approach it? Simple, I would consider the context. Which in this case, is the <strong>readership</strong>.</p><p>Imagine a machine learning model that isn’t just based on the contents of the story, but takes into account where the story is shared, and by whom. That’s immediately a more compelling picture (to me, at least).</p><p>With a model like that, you could infer the political leanings of readers for individual stories, rather than blanket labeling a publisher as left or right. You could put together a social graph like this (inspired by <a href="https://help.tumblr.com/hc/en-us/articles/15540197782551-Reblog-Graph">tumblr’s reblog graph</a> feature) for each story, to find where discussion of stories is happening, and what they’re saying about it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bMHaQ0JNEkeBeHIf" /><figcaption>Mock of social graph for a bellingcat story (this is fake data!)</figcaption></figure><p>I was inspired to consider this approach from an old, now deleted, <a href="https://web.archive.org/web/20211124014240/https://twitter.com/shaunjcammack/status/1298722815349149699">tweet thread</a> by Shaun Cammack on the Kenosha riots. In the thread, Cammack catalogs how the same story was interpreted by left-leaning readers and right-leaning readers, and which facts each group tended to index on. I wrote about it in my <a href="https://medium.com/@vastava/media-bias-charts-oversimplify-a-complex-non-issue-7fdddd78a4f1">mini-rant about Media Bias charts</a> earlier this week.</p><p>I think this kind of study would provoke more productive discussion than the current narrative of “journalist bias bad.” And if it exists, please let me know.</p><p>—</p><p>Okay, I wrote this several months ago but decided against publishing because it seemed overly ranty. And the MIT study I harped on was published in 2021, so I thought this methodology was on its way out… until I saw <a href="https://www.ft.com/content/e577411e-3bf2-4fb4-872a-8b7d5e9139d3">this story in the Financial Times</a>. Sentiment analysis, in the year 2024. And on books published 1600–1900 (?!!?!?).</p><p>Those charts hinge on a flawed methodology — what you’re probably seeing is English modernizing towards what we know it today. But instead of writing a new essay in response to it, I’m repurposing old garbage. Because I can also be lazy! But at least I’m not getting paid for it (<a href="https://amandaclaypool.medium.com/dear-medium-your-new-partner-program-sucks-85aeaf28890b#:~:text=Medium%20now%20compensates%20for%20engagement,an%20article%20doesn&#39;t%20count.">thanks, medium</a>).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adaed698f955" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Media Bias” charts oversimplify a complex non-issue]]></title>
            <link>https://vastava.medium.com/media-bias-charts-oversimplify-a-complex-non-issue-7fdddd78a4f1?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/7fdddd78a4f1</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[media-bias]]></category>
            <category><![CDATA[fact-checking]]></category>
            <category><![CDATA[journalism]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Sat, 20 Jan 2024 15:50:43 GMT</pubDate>
            <atom:updated>2024-01-20T15:50:43.667Z</atom:updated>
            <content:encoded><![CDATA[<p>This goddamn chart:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9_BavvzqpattUQCJjLJQg.jpeg" /><figcaption>Yes, Medium is social media don’t @ me. Source: <a href="https://adfontesmedia.start.page/">https://adfontesmedia.start.page/</a></figcaption></figure><p>I hate this thing. Twice per year, Ad Fontes Media releases these charts to help readers assess the “reliability” and “bias” of various media sources.</p><p>Because yes, trust in media is at an <a href="https://www.axios.com/2023/10/24/americans-trust-in-media-plummets-to-historic-low-poll">all-time low</a>. But does that have anything to do with journalists’ “bias?” Trust in <a href="https://www.forbes.com/sites/emilywashburn/2023/02/03/america-less-confident-in-police-than-ever-before-a-look-at-the-numbers/?sh=17cdeb256afb">almost</a> <a href="https://thehill.com/blogs/blog-briefing-room/news/3514131-trust-in-government-near-all-time-low-poll/">everything</a> <a href="https://www.insidehighered.com/news/business/financial-health/2023/07/11/american-confidence-higher-ed-hits-historic-low">is at</a> an all-time low. That’s just the nature of how people interact with institutions now.</p><p>These types of charts put all the onus on journalists and editors and completely remove readers from the equation. Which makes sense in some ways — newsrooms should be considered the authority, and the guy with a Simpsons’ profile picture on Twitter/X shouldn’t have any sway over how a story is written.</p><p>But, readers do control how stories are interpreted. Shaun Cammack has an amazing, now deleted, <a href="https://web.archive.org/web/20211124014240/https://twitter.com/shaunjcammack/status/1298722815349149699">tweet thread</a> on the Kenosha riots. In the thread, Cammack catalogs how the same story was interpreted by left-leaning readers and right-leaning readers, and which facts each group tended to hook onto. That story could come from one of the publishers in the center of Ad Fontes’ chart and be totally neutral in its language, if that’s even possible, but readers will still find a way to take it to a biased conclusion. Because duh.</p><p>The British Broadcasting Corporation, or BBC, is in the middle of this upside-down horseshoe so let’s take a closer look at it. Do you consider the BBC to be neutral?</p><p>In a 2018 <a href="https://www.bmgresearch.co.uk/is-the-bbc-biased-bmg-reveals-public-perceptions-of-broadcaster-impartiality-in-the-uk/">survey</a>, 40 percent of respondents considered it to be politically biased in some way. To the shock of no one, I’m sure. But what do those respondents mean exactly? Is the BBC biased towards the political left? The right? The Labour party, the Tories or a secret third thing? Well… it depends who you’re asking.</p><p>Because in that survey, 18 percent of respondents said the BBC is biased towards the right; but 22 percent of them believed that it was biased towards the opposite. So, net-net, you could consider the BBC to be slightly more biased towards the left. I guess.</p><p>Concluding that the BBC is biased based on these numbers is a hollow reading. The difference in people believing the BBC is left-leaning versus right-leaning is so marginal that it may as well be considered center. So maybe the Ad Fontes chart is correct. But that’s also not the full picture, right?</p><p>The truth is this survey is not a measurement of how biased the BBC is, even though that’s how it’s presented. It’s a measurement of how people <strong>perceive</strong> the BBC. And that’s arguably a more important metric than some arbitrary “media bias” score, but it’s almost never talked about. I mean, I had to go back to 2018 to even find a survey that summarized the results in this way. (Never mind the fact that Google sucks at surfacing relevant search results now…)</p><p>There’s another group — 24 percent — that don’t know if the BBC is biased one way or another, and 37 percent believed it to be truly neutral. Regardless, the largest group believed to be biased <strong>in some way</strong>, so that’s what the <a href="https://www.bmgresearch.co.uk/wp-content/uploads/2018/05/BMG-Research-BBC-bias-press-release.pdf">headlines</a> will say.</p><p>The problem of “media bias” is overstated, if it even is one. These types of charts present a false dichotomy between journalistic reliability and partisanship.</p><p>/end rant</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7fdddd78a4f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Essay 0]]></title>
            <link>https://vastava.medium.com/essay-0-329abcb58fbb?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/329abcb58fbb</guid>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Sun, 31 Dec 2023 14:58:17 GMT</pubDate>
            <atom:updated>2023-12-31T14:58:17.743Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey y’all. I used to post data science and vaguely technical tutorials here, but had a long hiatus. I’m (thinking of) coming back to this platform, but maybe not in the way you might expect.</p><p>I’ve been writing again. Mostly essays to help me think through my feelings, reactions to external events and general motivations. It’s been very healthy for me.</p><p>And also, I’ve realized that I’m terrible at it! My writing muscles have completely atrophied. And there’s no better way to tone those muscles than to start writing publicly again.</p><p>I’m not sure what this space will be. For now I’m thinking essays about tech and how it intersects with culture, behavior, blah blah blah. What I’m trying to say is that I have training wheels on — expect fluidity.</p><p>It felt weird to post without saying nothing, so enter Essay 0. A low stakes re-introduction back into this space. I know this is entirely superfluous and few people will care. I know this is kind of embarrassing. But also writing is embarrassing; and I have to put myself in that headspace and be okay with it.</p><p>Catch y’all in 2024.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=329abcb58fbb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to scrape IMDB to determine overlap between casts and crews]]></title>
            <link>https://vastava.medium.com/how-to-scrape-imdb-to-determine-overlap-between-casts-and-crews-57e659565ffd?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/57e659565ffd</guid>
            <category><![CDATA[network-analysis]]></category>
            <category><![CDATA[scraping]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[imdb]]></category>
            <category><![CDATA[web-scraping]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Sat, 07 May 2022 14:47:39 GMT</pubDate>
            <atom:updated>2022-05-07T14:47:39.855Z</atom:updated>
            <content:encoded><![CDATA[<h4>Analyzing the recent trend of superhero shows, and the people working on them</h4><p>I can’t be the only one who’s watched a TV show, and was eerily reminded of another show or movie. For me, the latest instance of this came when I was watching <em>Doom Patrol </em>on HBO Max, and was picking up some vibes that were very similar to that of Netflix’s <em>Umbrella Academy</em>.</p><p>I wasn’t the only one to see the <a href="https://www.nytimes.com/2019/02/14/arts/television/umbrella-academy-doom-patrol-review.html">similarities between these two shows</a>, and after some digging, I found that there were a number of producers that worked on both <em>Doom Patrol </em>and <em>Umbrella Academy. </em>That of course explains why these shows have similar aesthetics and character moments.</p><p>The data scientist in me then got an idea. What if we could use data to determine overlap between the cast/crew of a television show, and use that as a proxy to analyze similarities between the shows?</p><p>As always, the code for this project lives in my “<a href="https://github.com/vastava/data-science-projects">data-science-projects</a>” repo on Github. The specific folder for this project is <a href="https://github.com/vastava/data-science-projects/tree/master/tv-cast-overlap">here</a>.</p><p>I developed this project for a Youtube video analyzing similarities between “surreal superhero shows,” which you can watch below!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FoawgzOHSYZI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoawgzOHSYZI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FoawgzOHSYZI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9984b7a714cda4cdd1974316da5fb9f8/href">https://medium.com/media/9984b7a714cda4cdd1974316da5fb9f8/href</a></iframe><h3>Scraping the Data</h3><p>The first step in this project is to procure a list of all of the people who worked on a given show. IMDB makes this very easy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2s1oUR0EgoaMPXF4RBYScA.png" /><figcaption>Screenshot of IMDB credits for Doom Patrol.</figcaption></figure><p>IMDB has a “full cast and crew” page for each of the television shows in their database (<a href="https://www.imdb.com/title/tt8416494/fullcredits">here’s</a> the link for <em>Doom Patrol</em>). This list is handy because it buckets all of the people into their roles (ie. writing/cast/makeup, etc.) Note that people can be listed multiple types if they have more than one role on a given production.</p><p>Scraping this information is relatively straightforward since the page is constructed using HTML tables. Below is the function I used to scrape this data:</p><pre><strong>def</strong> scrapeCrew(url): <em>#scrape cast and crew from imdb</em><br>    html = urlopen(url)<br>    soup = BeautifulSoup(html, &#39;html.parser&#39;)<br>    tables = soup.find_all(&#39;table&#39;)<br>    headers = soup.find_all(&#39;h4&#39;)<br>    dfs = []<br>    <strong>for</strong> i <strong>in</strong> range(len(tables)):<br>        header = headers[i].text.strip().replace(&#39;\n&#39;, &#39;&#39;)<br>        rows = tables[i].find_all(&#39;tr&#39;)<br>        names = []<br>        roles = []<br>        presence = []<br>        <strong>if</strong> header == &#39;Series       Cast&#39;:<br>            <strong>for</strong> row <strong>in</strong> rows[1::2]:<br>                names.append(row.find_all(&#39;a&#39;)[1].text)<br>                roles.append(row.find_all(&#39;a&#39;)[2].text)<br>                presence.append(row.find_all(&#39;a&#39;)[3].text)<br>            df = pd.DataFrame() <br>            df[&#39;name&#39;] = names<br>            df[&#39;role&#39;] = roles<br>            df[&#39;presence&#39;] = presence<br>            dfs.append(df)<br>        <strong>else</strong>:<br>            <strong>for</strong> row <strong>in</strong> rows:<br>                names.append(row.find(&#39;a&#39;).text)<br>                presence.append(row.find(&#39;td&#39;, class_=&#39;credit&#39;).text)<br>            df = pd.DataFrame() <br>            df[&#39;name&#39;] = names<br>            df[&#39;role&#39;] = header<br>            df[&#39;presence&#39;] = presence<br>            dfs.append(df)<br>    temp = pd.concat(dfs)<br>    temp[&#39;show&#39;] = soup.find(&#39;h3&#39;).text.strip()<br>    <strong>return</strong> temp</pre><p>It’s worth noting that IMDB displays the cast information differently than they do for every one else in the crew, which is why I wrote an if condition to cast whether or not the given header is “Cast.”</p><p>After feeding that function a URL, you’ll wind up with a dataframe that might look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xqwqsMHrS-gwpJiSUdlhhA.png" /><figcaption>DataFrame of Doom Patrol’s cast and crew.</figcaption></figure><p>Obviously, this dataframe will need to be cleaned (a simple string replacement for “\n” and any other special characters should do the trick).</p><p>The other thing you might notice is that I included a “show” column, even though all rows in the dataframe should have the same value (in this case, Doom Patrol). However, the purpose of this exercise is to determine cast overlap between <em>multiple</em> shows, so you will have to run this scrapeCrew function multiple times and then you will have to concatenate the dataframes using panda’s concat method. The show column will be useful when we have multiple shows in the dataframe.</p><h3>Determining Cast Overlap</h3><p>For my project, I scraped the cast and crew of <em>Doom Patrol, Wandavision, Umbrella Academy </em>and<em> Legion — </em>all of which are popular, surreal television shows based on comic book characters<em>.</em></p><p>As I said earlier, some people are listed multiple times in the credits of a show if they had multiple roles. We can use a combination of panda’s drop_duplicates and value_counts methods to determine a list of people who worked on multiple shows (rather than having multiple roles).</p><p>Below is the code I used to determine people who worked on more than one show out of <em>Doom Patrol, Wandavision, Umbrella Academy </em>and<em> Legion. </em>The variable “all_shows” refers to my dataframe of all cast and crew members.</p><pre>all_shows[[&quot;name&quot;, &quot;show&quot;]].drop_duplicates()[&quot;name&quot;].value_counts().head(20) <em>#drop duplicates due to multiple roles</em></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*P8uAblXJ4pCRwm4FgI5kNA.png" /><figcaption>Pandas series object of people who worked on multiple shows.</figcaption></figure><p>We can also group our “all_shows” dataframe by the “name” column to achieve a similar effect. This way we will not only have the names of all of the people who worked on multiple shows, but will also retain the show and role information as well.</p><pre>grouped = all_shows.groupby(&quot;name&quot;).agg(<strong>lambda</strong> x: pd.unique(x))</pre><p>The aggregator function calls panda’s unique method so that we keep all information for each name. As you can see in the below screenshot, the “role”, “presence” and “show” columns for Aaron Shore now have arrays as a value instead of an individual string. This is because Aaron Shore worked on both <em>Doom Patrol</em> and <em>Wandavision</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zgQs3cEvRuAl-LjtNKM3aQ.png" /><figcaption>DataFrame grouped by name.</figcaption></figure><p>I also created the “num_shows” column by calculating the length of the array in the “show” column. However, if you take a look at the screenshot, you will definitely see that something wacky is going on with this “num_shows” column.</p><p>And that is because the “show” column has mixed types. If a person has only worked on 1 show, the value’s type is string. Otherwise, it is an array. Now because we used the python function len to calculate “num_shows,” if the value type is a string, then “num_shows” represents how many characters is in the string, rather than how many shows the person worked on.</p><p>There are ways around this. However, I was only interested in people who worked on multiple shows so I ended up filtering out any row that had a num_shows value greater than 4, anyway.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pPAhWjxtErvtinVJr-FeNw.png" /><figcaption>DataFrame of crew members who worked on multiple shows out of my itemset (Doom Patrol, Wandavision, Legion and Umbrella Academy).</figcaption></figure><p>What you’re left with is a DataFrame of people who worked on multiple shows out of the ones you’ve specified (for me, that was <em>Doom Patrol</em>, <em>Wandavision</em>, <em>Legion</em> and <em>Umbrella Academy</em>).</p><p>You can do some pretty interesting network analysis with this type of data, or even look at how the roles differ between shows for certain people (eg. maybe they directed an entire season of one show, and only guest featured in another). The options are pretty endless here!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57e659565ffd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to scrape Rotten Tomatoes for historic Tomatometer scores]]></title>
            <link>https://vastava.medium.com/how-to-scrape-rotten-tomatoes-for-historic-tomatometer-scores-426f01a55a0d?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/426f01a55a0d</guid>
            <category><![CDATA[scraping]]></category>
            <category><![CDATA[movies]]></category>
            <category><![CDATA[rotten-tomatoes]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[wonder-woman-1984]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Thu, 18 Feb 2021 15:54:07 GMT</pubDate>
            <atom:updated>2021-02-19T10:15:36.368Z</atom:updated>
            <content:encoded><![CDATA[<h4>Leverage the Wayback Machine’s archived URLs for time series analysis of movie reviews</h4><p>I, like many others, was fascinated by how <em>Wonder Woman 1984’s</em> Rotten Tomatoes score fell from “certified fresh” to “rotten” over the course of a few weeks. I wanted to scrape Rotten Tomatoes for historic Tomatometer scores to see if this kind of drop in ratings can be seen for other films.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*rfAIxbiLsCQQQ_txkz4wLQ.png" /><figcaption>Image by author.</figcaption></figure><p>There are two different methods to scrape these historic reviews, and there are pros and cons to each. The first method is to scrape all of the reviews for a film, and then use each review’s date to calculate the Rotten Tomatoes score for that day. The second method is to use the Wayback machine to scrape historic scores from archived versions of the page.</p><p>The first method is certainly faster, but the second is more accurate (although might result in missing data for a few days). I ended up going for the second method for my analysis, but will walk you through both methods in this tutorial.</p><p>I scraped this data for a broader analysis of Rotten Tomatoes reviews. My video below summarizes my findings!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FOMwlxH5p6A4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DOMwlxH5p6A4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FOMwlxH5p6A4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7be21db2342ce459804f9a2ec1b8489/href">https://medium.com/media/d7be21db2342ce459804f9a2ec1b8489/href</a></iframe><p>The complete code for this project can be seen <a href="https://github.com/vastava/data-science-projects/tree/master/rotten-tomatoes">here</a>.</p><h4>Method #1: Scraping Rotten Tomatoes reviews</h4><p>The code for the first method can be seen in my <a href="https://github.com/vastava/data-science-projects/blob/master/rotten-tomatoes/rotten%20tomatoes%20reviews%20scraper.ipynb">review scraper Jupyter Notebook</a>.</p><p>To reiterate, this first method scrapes all critic reviews for a given film on Rotten Tomatoes, and uses those reviews to calculate the Tomatometer score for each day. This method can be inaccurate for a couple of reasons:</p><ol><li>Rotten Tomatoes typically waits 1–2 days after reviews start pouring in before publishing the Tomatometer score as a number they can “certify.” This method has no way of telling when that certification occurs, so the calculated score may be inaccurate in the first few days.</li><li>This method requires scraping reviews from multiple pages, and it’s possible that it misses reviews or pages, which would affect the score calculation. For the most part, the scraper is comprehensive but I have seen it miss a few reviews a couple of times which is why I ended up going with method #2.</li></ol><p>Still, I think this code is useful for scraping the *text* of reviews, especially if you’re interested in doing some type of sentiment analysis on film reviews.</p><p>I first defined regex patterns for each of the elements I was interested in scraping (eg. page numbers, reviews, ratings, etc.) and then created a make_soup function to make a beautifulsoup object out of a url.</p><pre><em>#regex patterns</em><br>page_pat = re.compile(r&#39;Page 1 of \d+&#39;)<br>review_pat = re.compile(r&#39;&lt;div class=\&quot;the_review\&quot; data-qa=\&quot;review-text\&quot;&gt;[;a-zA-Z\s,-.<strong>\&#39;</strong>\/\?\[\]\&quot;:<strong>\&#39;</strong>]*&lt;/div&gt;&#39;)<br>rating_pat = re.compile(r&#39;Original Score:\s([A-Z](\+|-)?|\d(.\d)?(\/\d)?)&#39;)<br>fresh_pat = re.compile(r&#39;small\s(fresh|rotten)\&quot;&#39;)<br>critic_pat = re.compile(r&#39;\/\&quot;\&gt;([A-Z][a-zA-Z]+\s[A-Z][a-zA-Z\-]+)|([A-Z][a-zA-Z.]+\s[A-Z].?\s[A-Z][a-zA-Z]+)|([A-Z][a-zA-Z]+\s[A-Z]+<strong>\&#39;</strong>[A-Z][a-zA-Z]+)&#39;)<br>publisher_pat = re.compile(r&#39;\&quot;subtle\&quot;&gt;[a-zA-Z\s,.\(\)<strong>\&#39;</strong>\-&amp;;!\/\d+]+&lt;/em&gt;&#39;)<br>date_pat = re.compile(r&#39;[a-zA-Z]+\s\d+,\s\d+&#39;)</pre><pre><strong>from</strong> <strong>bs4</strong> <strong>import</strong> BeautifulSoup<br><strong>from</strong> <strong>requests</strong> <strong>import</strong> TooManyRedirects<br><strong>import</strong> <strong>re</strong><br><br><strong>def</strong> make_soup(url):<br>    <strong>try</strong>:<br>        r = requests.get(url)<br>        soup = BeautifulSoup(r.content, &#39;html.parser&#39;)<br>    <strong>except</strong> TooManyRedirects:<br>        soup = &#39;&#39;<br>    <strong>return</strong> soup</pre><p>Using these regex patterns, I then defined two helper functions that take in a beautifulsoup object: one to get the number of pages of a review, and another to get the reviews from each page.</p><pre><strong>def</strong> get_num_pages(soup):<br>    match = re.findall(page_pat,str(list(soup)))<br>    <strong>if</strong> len(match) &gt; 0:<br>        match = match[0]<br>        match = match.split(&#39; of &#39;)[-1]<br>        <strong>return</strong> match<br>    <strong>else</strong>:<br>        <strong>return</strong> <strong>None</strong></pre><pre><strong>def</strong> get_critic_reviews_from_page(soup):<br>    reviews = list()<br>    rating = list()<br>    fresh = list()<br>    critic = list()<br>    top_critic = list()<br>    publisher = list()<br>    date = list()<br>    <br>    soup = str(soup)<br>    review_soup = soup.split(&#39;=&quot;review_table&#39;)[1].split(&#39;row review_table_row&#39;)<br>    review_soup.pop(0)<br><br>    <strong>for</strong> review <strong>in</strong> review_soup:<br>        match = re.findall(review_pat, str(review))<br>        <strong>if</strong> len(match) &gt; 0:<br>            m = match[0]            <br>            <strong>for</strong> iden <strong>in</strong> [&#39;&lt;div class=&quot;the_review&quot; data-qa=&quot;review-text&quot;&gt; &#39;,&#39;&lt;/div&gt;&#39;]:<br>                m = m.replace(iden,&#39;&#39;)<br>            reviews.append(m.strip(&#39;&quot;&#39;))            <br>            <em># extract rating</em><br>            match = re.findall(rating_pat, str(review))<br>            <strong>if</strong> len(match) &gt; 0:<br>                m = match[0][0]<br>                <strong>if</strong> &#39;/1&#39; <strong>in</strong> m:<br>                    sp_m = m.split(&#39;/&#39;)<br>                    <strong>if</strong> sp_m[-1] == &#39;1&#39;:<br>                        sp_m[-1] = &#39;10&#39;<br>                    m = &#39;/&#39;.join(sp_m)<br>                rating.append(m)<br>            <strong>else</strong>:<br>                rating.append(<strong>None</strong>)            <br>            <em># extract fresh indicator</em><br>            match = re.findall(fresh_pat, str(review))<br>            <strong>if</strong> len(match) &gt; 0:<br>                fresh.append(match[0])<br>            <strong>else</strong>:<br>                fresh.append(<strong>None</strong>)            <br>            <em># extract critic</em><br>            match = re.findall(critic_pat, str(review))<br>            <strong>if</strong> len(match) &gt; 0:<br>                critic.append(&#39;&#39;.join(match[0]))<br>            <strong>else</strong>:<br>                critic.append(<strong>None</strong>)            <br>            <em># check if top critic</em><br>            <strong>if</strong> &#39;&gt; Top Critic&lt;&#39; <strong>in</strong> str(review):<br>                top_critic.append(1)<br>            <strong>else</strong>:<br>                top_critic.append(0)            <br>            <em># extract publisher</em><br>            match = re.findall(publisher_pat, str(review))<br>            <strong>if</strong> len(match) &gt; 0:<br>                m = match[0]<br>                m = m.replace(&#39;&quot;subtle&quot;&gt;&#39;, &#39;&#39;)<br>                m = m.replace(&#39;&lt;/em&gt;&#39;,&#39;&#39;)<br>                publisher.append(m)<br>            <strong>else</strong>:<br>                publisher.append(<strong>None</strong>)            <br>            <em># extract date</em><br>            match = re.findall(date_pat, str(review))<br>            <strong>if</strong> len(match) &gt; 0:<br>                date.append(match[0].strip(&#39;&quot;&#39;))<br>            <strong>else</strong>:<br>                date.append(<strong>None</strong>)<br>            <br>    <strong>return</strong> [reviews, rating, fresh, critic, top_critic, publisher, date]</pre><p>Once I had those functions defined, I could get to scraping. I defined a get_critic_reviews function that calls those helper functions and takes in a Rotten Tomatoes url (eg. <a href="https://www.rottentomatoes.com/m/wonder_woman_1984/">https://www.rottentomatoes.com/m/wonder_woman_1984/</a>)</p><pre><strong>def</strong> get_critic_reviews(page):<br>    info = [[],[],[],[],[],[],[]]<br>    soup = make_soup(page + &quot;reviews&quot;)<br><em>#     print(soup)</em><br>    pages = get_num_pages(soup)<br><em>#     print(pages)</em><br>    <strong>if</strong> pages <strong>is</strong> <strong>not</strong> <strong>None</strong>:<br>        <strong>for</strong> page_num <strong>in</strong> range(1,int(pages)+1):<br>            soup = make_soup(page + &quot;reviews?page=&quot; + str(page_num) + &quot;&amp;sort=&quot;)<br>            c_info = get_critic_reviews_from_page(soup)<br>            <br>            <em># accumulate review info</em><br>            <strong>for</strong> i <strong>in</strong> range(len(c_info)):<br>                info[i] = info[i] + c_info[i]<br>        <br>        c_info = dict()<br>        keys = [&#39;reviews&#39;, &#39;rating&#39;, &#39;fresh&#39;, &#39;critic&#39;, &#39;top_critic&#39;, &#39;publisher&#39;, &#39;date&#39;]<br>        <strong>for</strong> k <strong>in</strong> range(len(keys)):<br>            c_info[keys[k]] = info[k]        <br>    <strong>else</strong>:<br>        c_info = <strong>None</strong>        <br>    <strong>return</strong> c_info</pre><p>This function should return a dictionary, which you can then convert into a dataframe using pd.DataFrame.from_dict(). Your dataframe should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eMXyIGPGyeVv-zIro2Af7Q.png" /><figcaption>Image by author.</figcaption></figure><p>Now obviously this dataframe needs to be cleaned a bit (especially the reviews column). I wasn’t interested in doing text analysis, but if I was, I would simply add “.text” to the div element in the scraper.</p><p>What <em>I am</em> interested in is somehow converting the “fresh” column to integers {1,0} so I can calculate the cumulative Rotten Tomatoes score for each day. And this is pretty easy to do in pandas.</p><pre>all_films[&#39;score&#39;] = all_films[&#39;fresh&#39;].apply(<strong>lambda</strong> x: 1 <strong>if</strong> x == &#39;fresh&#39; <strong>else</strong> 0)</pre><p>We can now use this “score” column to <em>finally</em> calculate the Rotten Tomatoes score! I first grouped the reviews by date, and used two aggregator functions on the score column (sum and count). This way, we can take the cumulative sums of scores and divide it by the cumulative count of scores.</p><p>Essentially, we are calculating what percent of total critics rated the film fresh on each day, which is how the Tomatometer score is calculated.</p><pre>df = all_films[all_films[&#39;Film&#39;] == &quot;Wonder Woman 1984&quot;]<br>grouped_1 = df[[&#39;date&#39;, &#39;score&#39;]].groupby(&#39;date&#39;).agg([sum, &#39;count&#39;])<br>grouped_1.columns = grouped_1.columns.droplevel(0)<br>grouped_1.cumsum()[&#39;sum&#39;]/grouped_1.cumsum()[&#39;count&#39;]</pre><p>And presto! You should get a series object like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*clsiyNCrO2BChszsR-amdw.png" /><figcaption>Image by author.</figcaption></figure><h4>Method #2: Scraping Rotten Tomatoes scores from the Wayback Machine</h4><p>The code for the second method can be seen in my <a href="https://github.com/vastava/data-science-projects/blob/master/rotten-tomatoes/rotten%20tomatoes%20score%20scraper.ipynb">score scraper Jupyter Notebook</a>.</p><p>The second method may seem more obvious, but it is much more complicated. The Rotten Tomatoes website has changed a lot in the past 3 years, and because of that I had to write multiple scrapers to account for the different versions of the website.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*GfW_5GIFW187oHhJvZHCzg.png" /><figcaption>Image by author.</figcaption></figure><p>I’ve only verified that these scrapers work for archived websites from 2018 onwards. If you’re going with this method, make sure you’re using the correct scraper for the correct year!</p><p>These scrapers make use of the <a href="https://pypi.org/project/waybackpy/">waybackpy Python package</a>, which allows you to access archived versions of a URL programmatically. I wrote functions to scrape the number of critic and audience reviews, and also the critic and audience scores from each URL.</p><p>Good news is the code for this method is much shorter! Here’s the function that scrapes the number of reviews for critics and audiences:</p><pre><em>#get number of critic and audience reviews</em><br><strong>def</strong> getNumReviews(soup):        <br>    critic = soup.find_all(&quot;small&quot;, class_=&quot;mop-ratings-wrap__text--small&quot;)<br>    audience = soup.find_all(&quot;strong&quot;, class_=&quot;mop-ratings-wrap__text--small&quot;)<br>    <strong>if</strong> critic:<br>        critic = critic[0].text.replace(&quot;<strong>\n</strong>&quot;, &#39;&#39;).strip()<br>    <strong>else</strong>:<br>        critic = soup.find_all(&quot;a&quot;, class_=&#39;scoreboard__link scoreboard__link--tomatometer&#39;)<br>        critic = critic[0].text<br>    <strong>if</strong> len(audience) &gt; 1:<br>        audience = audience[1].text.replace(&quot;Verified Ratings: &quot;, &#39;&#39;)<br>    <strong>else</strong>:<br>        audience = soup.find_all(&quot;a&quot;, class_=&#39;scoreboard__link scoreboard__link--audience&#39;)<br>        <strong>if</strong> audience:<br>            audience = audience[0].text<br>        <strong>else</strong>:    <br>            audience = &quot;Coming&quot;<br>    <strong>return</strong> [critic, audience]</pre><p>And here’s the function(s) to scrape the audience and critic scores:</p><pre><em>#2018 scraper</em><br><strong>def</strong> get_score(soup):<br>    critic = soup.find(&#39;span&#39;, {&#39;class&#39;: &quot;meter-value superPageFontColor&quot;})<br>    audience = soup.find(&#39;div&#39;, {&#39;class&#39;: &quot;audience-score meter&quot;}).find(&#39;span&#39;, {&#39;class&#39;: &quot;superPageFontColor&quot;})<br>    <strong>return</strong> [critic.text, audience.text]</pre><pre><em>#2019-21 scraper</em><strong><br>def</strong> getScore(soup):<br>    temp = soup.find_all(&#39;div&#39;, class_=&#39;mop-ratings-wrap__half&#39;)<br>    <strong>try</strong>:<br>        critic = temp[0].text.strip().replace(&#39;<strong>\n</strong>&#39;, &#39;&#39;).split(&#39; &#39;)[0] <em>#2020 scraper        </em><br>        <strong>if</strong> len(temp) &gt; 1:<br>            audience = temp[1].text.strip().replace(&#39;<strong>\n</strong>&#39;, &#39;&#39;).split(&#39; &#39;)[0]<br>        <strong>else</strong>:<br>            audience = &quot;Coming&quot;<br>    <strong>except</strong>:<br>        scores = soup.find(&quot;score-board&quot;) <em>#2021 scraper</em><br>        <strong>return</strong> [scores[&quot;tomatometerscore&quot;], scores[&quot;audiencescore&quot;]]<br>    <strong>return</strong> [critic, audience]</pre><p>One slightly annoying thing is that the format for the scores vary slightly across different versions of the Rotten Tomatoes website. Sometimes there will be a ‘%’ attached to the string, sometimes it will have ‘score’ or other related words included with it. Make sure to look out for those edge cases when cleaning the data!</p><p>Anyhow, after defining the scraping helper functions, we can scrape the data! I wrote a build function that takes in a url, and a starting and end date to scrape the data for.</p><pre><strong>def</strong> build(key, date1, date2):<br>    dates = pd.date_range(date1, date2).tolist()<br>    wayback = waybackpy.Url(movie_urls[key], user_agent)<br>    scores = []<br>    <strong>for</strong> date <strong>in</strong> dates:<br>        <strong>try</strong>:<br>            archived = wayback.near(year=date.year, month=date.month, day=date.day).archive_url<br>        <strong>except</strong>:<br>            print(date)<br>            <strong>continue</strong><br>        page = requests.get(archived).text<br>        soup = BeautifulSoup(page, &#39;lxml&#39;)<br>        scores.append(getScore(soup) + getNumReviews(soup) + [date, key])<br>    <strong>return</strong> scores</pre><p>The output of this function will be an array of arrays, like in the below image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*Q9-IhZTs05JW5dMscO1XXA.png" /><figcaption>Image by author.</figcaption></figure><p>Again, we can easily convert this output into a DataFrame for our analysis!</p><pre>pd.DataFrame(df, columns=[&quot;critic&quot;, &quot;audience&quot;, &quot;criticNum&quot;, &quot;audienceNum&quot;, &quot;date&quot;, &quot;film&quot;]))</pre><p>The pitfall of using this method is that if the Wayback machine has not archived the URL for a given day, you will not be able to scrape the data for that day. I tried this method for upwards of 20 films and it generally wasn’t a problem. There might be 1–5 days of missing data depending on your date range, but if you’re looking to scrape scores for a more obscure film that isn’t likely to be archived, you might have to go with method #1.</p><p>And finally, check out my video if you’re interested in hearing what I found from analyzing this data!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FOMwlxH5p6A4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DOMwlxH5p6A4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FOMwlxH5p6A4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7be21db2342ce459804f9a2ec1b8489/href">https://medium.com/media/d7be21db2342ce459804f9a2ec1b8489/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=426f01a55a0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scraping Wikipedia Page Views to Make a 2020 Rewind]]></title>
            <link>https://medium.com/data-science/scraping-wikipedia-page-views-to-make-a-2020-rewind-c9bcac97fa38?source=rss-c1df9a412b37------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9bcac97fa38</guid>
            <category><![CDATA[scraping]]></category>
            <category><![CDATA[wikipedia]]></category>
            <category><![CDATA[2020-recap]]></category>
            <category><![CDATA[page-views]]></category>
            <category><![CDATA[rewind]]></category>
            <dc:creator><![CDATA[Anjali Shrivastava]]></dc:creator>
            <pubDate>Tue, 26 Jan 2021 18:42:54 GMT</pubDate>
            <atom:updated>2021-01-26T18:52:19.875Z</atom:updated>
            <content:encoded><![CDATA[<h4>Looking back through the lens of Wikipedia’s most popular articles</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dAOA8EsC1rw_Oe30ygWxvQ.png" /><figcaption>Image by author.</figcaption></figure><p>People often go to Wikipedia to make sense of current events — perhaps to get a synopsis of Netflix’s latest release, or to read about the achievements of a recently deceased celebrity.</p><p>I had the idea to use Wikipedia page view data to create a “2020 rewind” of sorts — an animated timeline of current events and trends throughout the year of 2020. Here’s how I did it.</p><p>The code for this project can be accessed <a href="https://github.com/vastava/data-science-projects/tree/master/wikipedia-rewind">here</a>.</p><p>I first defined a “get_traffic” function to get the page view data for a given day:</p><pre>TOP_API_URL = &#39;https://wikimedia.org/api/rest_v1/metrics/pageviews/&#39;\               &#39;top/<strong>{lang}</strong>.<strong>{project}</strong>/all-access/<strong>{year}</strong>/<strong>{month}</strong>/<strong>{day}</strong>&#39;  </pre><pre><strong>def</strong> get_traffic(year, month, day):   <br>   TOP_API_URL.format(lang=&#39;en&#39;,project=&#39;wikipedia&#39;, year=year, month=month,day=day)resp = urllib2.urlopen(url)     <br>   resp_bytes = resp.read() <br>   data = json.loads(resp_bytes)     <br>   articles = data[&#39;items&#39;][0][&#39;articles&#39;]     <br>   <strong>return</strong> articles</pre><p>The call get_traffic(‘2020’, ‘02’, ‘21’), for example, would output a JSON object with the 1000 most viewed Wikpedia articles on that day. The object contains the article’s title, the number of views, and its “rank” for that day in terms of views.</p><p>I also created an is_article function to verify that each article is indeed an article before adding it to the dataframe.</p><pre><strong>def</strong> is_article(title, wiki_info):<br>    skip = [&#39;-&#39;, &#39;404.php&#39;, &#39;XHamster&#39;] + [wiki_info[&#39;mainpage&#39;]]<br>    prefixes = PREFIXES + wiki_info[&#39;namespaces&#39;]<br>    <strong>if</strong> title <strong>in</strong> skip:<br>        <strong>return</strong> <strong>False</strong><br>    <strong>if</strong> title == &quot;Media&quot;:<br>        <strong>return</strong> <strong>False</strong><br>    <strong>if</strong> title == &quot;Wikipedia&quot;:<br>        <strong>return</strong> <strong>False</strong><br>    <strong>if</strong> title == &quot;United_States_Senate&quot;:<br>        <strong>return</strong> <strong>False</strong><br>    <strong>for</strong> prefix <strong>in</strong> prefixes:<br>        <strong>if</strong> title.startswith(prefix + &#39;:&#39;):<br>            <strong>return</strong> <strong>False</strong><br>    <strong>return</strong> <strong>True</strong></pre><p>To create the rewind, I called the get_traffic function for every day in the year 2020 and concatenated the results into a dataframe. I only saved the 25 most viewed articles in this dataframe, rather than the entire output of get_traffic.</p><pre>date1 = &#39;2020-01-01&#39;<br>date2 = &#39;2021-01-07&#39;<br>dates = pd.date_range(date1, date2).tolist()</pre><pre>top_array = []<br><br><strong>for</strong> date <strong>in</strong> dates:<br>    year=date.strftime(&#39;%Y&#39;)<br>    month = date.strftime(&#39;%m&#39;)<br>    day = date.strftime(&#39;<strong>%d</strong>&#39;)<br>    print(date)</pre><pre>    raw_traffic = get_traffic(year, month, day)<br>    articles = [a <strong>for</strong> a <strong>in</strong> raw_traffic <strong>if</strong> is_article(a[&#39;article&#39;], wiki_info)]<br>    top_array.append(articles[:25])</pre><pre>views_array = []<br>articles_array = []<br>ranks_array = []<br><br><strong>for</strong> i <strong>in</strong> range(len(top_array)):<br>    date = dates[i]<br>    array = top_array[i]<br>    <strong>for</strong> j <strong>in</strong> range(25):<br>        row = array[j]<br>        dates_array.append(date)<br>        articles_array.append(row[&#39;article&#39;].replace(&#39;_&#39;, &#39; &#39;))<br><em>#         print(articles_array)</em><br>        views_array.append(row[&#39;views&#39;])<br>        ranks_array.append(row[&#39;rank&#39;])</pre><pre>dict = {&#39;date&#39;: dates_array, &#39;article&#39;: articles_array, &#39;views&#39;: views_array, &#39;rank&#39;: ranks_array}<br>df = pd.DataFrame(data=dict)</pre><p>Once I had the dataset, I followed Mike Bostock’s tutorial to create a barchart race in d3.js with the data. I split the 2020 rewind into 2 parts to avoid an overly-long video.</p><p>If you watch both videos, you’ll notice some patterns — people seem most interested in politics, celebrity biographies, new entertainment releases, and the Bible. Articles like “Qasem Soleimani” or the “25th Amendment” will see a spike as people attempt to digest the daily news cycle. Similarly, when a famous person dies, many flock to Wikipedia to read their biography.</p><p>Enjoy the recap(s) below!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F39nV7f3HZg0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D39nV7f3HZg0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F39nV7f3HZg0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1a34fed16ff96ae8bfe633ba780c5aa/href">https://medium.com/media/b1a34fed16ff96ae8bfe633ba780c5aa/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FXP0Xft-MNqs%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DXP0Xft-MNqs&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FXP0Xft-MNqs%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/525c0962102de0aa2ad6729c658fc659/href">https://medium.com/media/525c0962102de0aa2ad6729c658fc659/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9bcac97fa38" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/scraping-wikipedia-page-views-to-make-a-2020-rewind-c9bcac97fa38">Scraping Wikipedia Page Views to Make a 2020 Rewind</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>