<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Trustworthy Social Media - Medium]]></title>
        <description><![CDATA[Our research group is working to understand how to improve conversations and discourse on social media, like Twitter and Facebook. By integrating cutting-edge computer science with quantitative social science, we are building a more trustworthy social media. - Medium]]></description>
        <link>https://medium.com/trustworthy-social-media?source=rss----ace6c515866f---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Trustworthy Social Media - Medium</title>
            <link>https://medium.com/trustworthy-social-media?source=rss----ace6c515866f---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:39:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/trustworthy-social-media" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Can You Label Less by Using Out-of-Domain Data?]]></title>
            <link>https://medium.com/trustworthy-social-media/can-you-label-less-by-using-out-of-domain-data-6e6e152b49df?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/6e6e152b49df</guid>
            <category><![CDATA[active-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Rafal Kocielnik]]></dc:creator>
            <pubDate>Sun, 05 Feb 2023 20:31:44 GMT</pubDate>
            <atom:updated>2023-02-05T20:31:44.768Z</atom:updated>
            <content:encoded><![CDATA[<p>Check out <a href="https://proceedings.mlr.press/v203/kocielnik23a/kocielnik23a.pdf">our paper</a> published on <a href="https://tl4nlp.github.io/">Workshop on Transfer Learning for Natural Language Processing</a> at NeurIPS 2022 conference</p><p>Social media propagates many toxic, biased and divisive comments. Despite significant advances in Machine Learning (ML), detecting such content is still very challenging. The definitions of what is considered toxic and the expressions of toxicity evolve fast and require careful judgment by skilled annotators. Unfortunately, labeling social-media data for dimensions of harmful content is challenging and labor-intensive. At the same time most ML approaches require vast amounts of labeled data to be effective.</p><h4>Active &amp; Transfer Learning with Few-shot Instructions (ATF) without any model fine-tuning!</h4><p>To reduce the annotation effort we combine several known techniques under one Active Transfer Few-shot Instructions (ATF) approach which is flexible to variable harmful content definitions, <strong>requires no training or fine-tuning</strong>, and can be executed near real-time. ATF (Fig 1) combines <a href="https://en.wikipedia.org/wiki/Transfer_learning">transfer learning</a> to leverage labels from existing labeled datasets, even if these were labeled under different definitions. It also uses <a href="https://en.wikipedia.org/wiki/Active_learning_(machine_learning)">active learning</a> techniques to work with limited annotations and request the most beneficial annotation proactively from human labeler. Finally, ATF leverages the already existing internal linguistic knowledge of large PLMs using natural text <a href="http://ai.stanford.edu/blog/understanding-incontext/">few-shot prompts</a>. A step-by-step overview of ATF is presented in Fig 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mh5NdHJXFZy2NQWrQltKdg.png" /><figcaption>Figure 1 — Overview of our Active Transfer Few-shot Instructions (ATF) framework for efficient labeling of text data.</figcaption></figure><h4>Application to Real-World #MeToo Twitter Dataset</h4><p>We applied our ATF framework to annotate a real-world <strong>#MeToo dataset</strong> collected from Twitter from January 2017 to September 2019 with 7.55 million tweets. We label this dataset for <em>“Sexually Explicit”</em> and <em>“Toxic”</em> dimensions. We transferred labels from existing human labeled datasets:</p><ul><li><strong>Social Bias Frames (SBIC)</strong><em> </em>[<a href="https://arxiv.org/pdf/1911.03891.pdf">Sap’20</a>] — the dataset contains 34k documents in the training set labeled under categories in which people project social biases and stereotypes onto others. It is labeled for <em>“Offensive”</em>, <em>“Lewd” </em>content with<em> </em>and meta dimensions on whether the harmful speech was intentional (<em>“Intent’’)</em> and whether it targets a particular social group (<em>“Group”).</em></li><li><strong>Hate Speech and Offensive Content Identification (HSOC)</strong> [<a href="https://dl.acm.org/doi/pdf/10.1145/3368567.3368584?casa_token=sYpPHb1sFhYAAAAA:LLiqTrbUySWBLeRvbpBfGb0i40dris4NAL-HLGYCfzn_XTajOMVcvoVG2ASYqyMIKDzzj7p0GnTWJxY">Mandl’19</a>] — the dataset contains 6k documents from Twitter and Facebook labeled for <em>“HOF”</em> whether a post contains hate, offensive, or profane content, <em>“Target”</em> whether a post contains an insult (targeted or untargeted).</li></ul><p>It is worth noting that both datasets have been annotated for dimensions of harmful content different that our intended labeling on #MeToo. Furthermore, the documents in these dataset are obtained from different social media platforms such as Reddit and Facebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WqF2alAnX3pYgR5Z-XXyOA.png" /><figcaption>Figure 2 — A) Positive and negative transfer from existing datasets (SBIC &amp; HASOC) to Perspective API labeled dimensions (“Sexually Explicit” and “Toxicity”) on a real-world dataset (#MeToo). The X-axis presents different model sizes of a <a href="https://arxiv.org/pdf/1909.08053.pdf">Megatron</a> model from NVIDIA. The Y-axis is the <a href="https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5">AUC metric</a> of classification performance, where the higher value is better. B) Relative impact of target annotation size on transfer effectiveness.</figcaption></figure><p>In Fig 2, we can see the results of using our ATF framework to transfer the labeling knowledge from these existing datasets to our target dimensions of interest on #MeToo. We can essentially see two scenarios: Positive and Negative transfer.</p><ul><li><strong>Positive transfer</strong> (#MeToo “Sexually Explicit”) — gains from transfer are sustained across model (Fig 2.A) and annotation sizes (Fig 2.B). As the model size increases, the gains in positive transfer tend to decrease only slightly (2.45% gap between gain from Meg1.3b and Meg22b) and the overall effectiveness of transfer is largely retained (B).</li><li><strong>Negative transfer</strong> (#MeToo “Toxicity”) — small gains can be inconsistent and turn into losses (1.91% gain in Meg1.3b, -3.3% loss for Meg8.3b, and a 0.9% gain for Meg22b).</li></ul><p>The main takeaways from these results are that: 1) if the positive or negative transfer occurs, it is retained across model (Fig 2.A)and target annotation sizes (Fig 2.B), 2) the higher initial baseline AUC for the models likely contributes to the negative transfer, 3) transfer effectiveness can increase with small target domain annotation size, but diminishes with an increasing number of annotations (Fig. B).</p><h4>Insights Into the Positive &amp; Negative Transfers</h4><p>We perform additional analysis to understand the nature of positive and negative transfers. Comparing the two scenarios, we can also see that the initial baseline performance of the models is consistently higher for the negative transfer scenario (mean AUC of 58.9) than for the positive one (mean AUC of 55.6). In fact the higher the initial performance of the PLM with a given annotation size (i.e., without source domain data) the lower the AUC gain from the transfer (ρ=−0.66).</p><p>We examine whether the sheer amount of external data from the source domain impacts transfer effectiveness. We find that the smaller HASOC dataset (6k) actually offers a higher mean gain of 7.54% compared to a much larger SBIC (34k) offering a mean gain of 4.76% in the same setup. We find that the difference in label imbalance between the source and target datasets is not correlated with AUC gain from the transfer (ρ=0.14). We also find that correlation between source and target labeling dimensions estimated on the source dataset (i.e., SBIC or HASOC) is only weakly related to AUC gain (ρ=−0.27).</p><p>The main takeaway is that negative transfer is more likely to happen if the initial PLM baseline on that task is higher. In other words, if PLMs internal knowledge is sufficient to perform well on the task, the value of supplying additional external information via transfer is limited.</p><h4>Limitations, Applications, and Future Work</h4><p>One limitation of our work is that the datasets we use rely on untrained crowd-sourced labeling which can be noisy and based on personal biases and perceptions [<a href="https://arxiv.org/pdf/1707.01477.pdf">Binns’17</a>]. Perspective API labeling has known limitations of its own [<a href="https://dl.acm.org/doi/pdf/10.1145/3442188.3445922">Bender’21</a>]. Furthermore, PLMs can be biased and toxic themselves when prompted [<a href="https://arxiv.org/pdf/2009.11462.pdf?fileGuid=zX3zH5DBtXQa0ktq">Gehman’20</a>], which also likely allows them to detect these dimensions based on their internal knowledge.</p><p>Some possible future applications of our ATF method:</p><ul><li>Noisy pre-labeling of unlabeled datasets and selecting samples for future fine-tuning (e.g., via disagreement-based active learning [11]).</li><li>With some limited initial human labeling of as few as 100 random documents, if the baseline few-shot performance is poor, using prelabeled out-of-domain data can improve the AUC without expending more human annotation effort.</li><li>We also plan to use this method to efficiently label custom dimensions of toxicity relevant to #MeToo and other real-world data, which are currently not supported by tools such as Perspective API.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e6e152b49df" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/can-you-label-less-by-using-out-of-domain-data-6e6e152b49df">Can You Label Less by Using Out-of-Domain Data?</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dynamics of Political Polarization: Insights from Using Machine Learning and Natural Language…]]></title>
            <link>https://medium.com/trustworthy-social-media/dynamics-of-political-polarization-surprising-stability-in-the-trump-era-congress-f21c2b8a98?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/f21c2b8a98</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[topic-modeling]]></category>
            <category><![CDATA[polarization]]></category>
            <category><![CDATA[dimensionality-reduction]]></category>
            <category><![CDATA[politics-and-elections]]></category>
            <dc:creator><![CDATA[Danny Ebanks]]></dc:creator>
            <pubDate>Fri, 04 Nov 2022 21:28:06 GMT</pubDate>
            <atom:updated>2023-01-04T21:52:24.635Z</atom:updated>
            <content:encoded><![CDATA[<h3>Dynamics of Political Polarization: Insights from Using Machine Learning and Natural Language Processing with Twitter Data</h3><p>The American public increasingly finds itself bitterly divided over political differences. Survey indicators, partisan media, and the public’s voting patterns inform this sense of division in our politics. That said, we use applications of Machine Learning and Natural Language Processing (NLP) methods in a novel way to paint a more nuanced picture of divisions in American political opinions.</p><p>It turns out that even very simple NLP methods that rely on simple word frequencies in politicians’ tweets can be extremely predictive when it comes to predicting party affiliation, getting over 80% accuracy without any special tuning. These simple models are very robust: a model trained on the tweets from the House of the Representatives can be equally predictive when tested on the tweets from the US Senators. Furthermore, even though politicians of both major political parties have become increasingly partisan and homogeneous in terms of their ideological commitments, these simple models can produce <strong>very</strong> credible rankings of the politicians along the left-right spectrum. These rankings very closely replicate similar rankings that have been based on politicians voting records. If these results withstand more rigorous scrutiny, NLP methods on politicians’ tweets could become a very simple measuring tool for ascertaining the degree of partisanship for any politician, even those with very meager or nonexistent voting records. In these settings, even simple assumptions on the structure of the topics yields large gains in understanding political polarization amongst political elites.</p><p>Imposing some additional latent structure on this bag-of-words approach can allow for interpretability on top of uncovering the original rankings. To this end, we use the Principal Components of the outputs from a Joint-Sentiment Topic model to produce scores of polarization. These scores describe relative positions of federal legislators, state legislators, members of the executive branch, local politicians, and journalists in a shared issue space over time. This allows us to analyze polarization in online speech across a variety of domains and over time. Based on posts from their public Twitter accounts from 2018 to 2022, we first uncover topics that politicians and journalists are discussing during this period. At the same time, we estimate whether sentiments underlying this discussion are positive, negative, or neutral. We then take these topic and sentiment scores to uncover online partisan positioning (using a more classical dimension reduction technique).</p><p>This method offers several advantages over existing methods in political science for measuring ideological positioning, such as <a href="https://voteview.com/about">DW-Nominate</a> (based on how legislators vote on bills) and <a href="https://data.stanford.edu/dime">Bonica Scaling</a> (based on campaign donations). First, it does not rely on binary data, so it is able to better identify moderates and extremists in the party. Second, the positions estimated by our scores are derived from continuous data, based on salient topics, allowing for coherent analysis of the variation contributing to each dimension. Third, it allows for comparisons for all Twitter or social media users in the same derived space.</p><p><strong>Figure 1: Discussion Space on Political Twitter</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NvgXkGZ_iny_Ese4" /></figure><p>With our method, we show in Figure 1 that members of the public can be directly compared with their politicians, enabling new ways to measure political representation. In Figure 1, we identify two dimensions to the data we find on political Twitter: a partisan dimension on the Y-Axis and a Journalistic Dimension on the X-Axis. The further up you go on the Y-Axis, the more liberal we find a speaker’s tweets on Twitter. The further right on the X-axis, the more like our method considers you to be a journalist. The topics contributing the most to political polarization in this period are tweets conveying a negative view of Trump Environmental Regulations for Democrats, and tweets conveying negative sentiment towards Mask Policies for Republicans. Unsurprisingly, we find that topics related to highlighting another tweet or article contribute the most to being identified as a journalist. Finally, the scores can be estimated tractably over arbitrarily long time periods, and so policy stances can be compared over time, in the same space.</p><p>We note journalists generally tend to be less ideologically cohesive than politicians in how they communicate on social media presence, according to our method. This finding could be an artifact of how journalists were placed in their own cluster, but it might also have implications for how Journalists exist in their own information ecosystem, one separate from political elites.</p><p>Some of our other preliminary results are surprising, especially given the heated tone of political news media.</p><p><strong>Figure 2: Dynamics in Polarization of Senate Twitter Accounts</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*62TU1-EdqI1WAU1w" /></figure><p>First, as we see in Figures 2 and 3, there are relatively stable levels of polarization in elite speech on Twitter during the 2022 to 2020 period. At the start of the Covid pandemic in March 2020, both the House and Senate depolarized, but quickly re-polarized thereafter. We note this period with the vertical line in both figures. We find strong statistical evidence this convergence in political speech was unlikely to be mere random noise. This suggests something akin to a rally-around-the-flag effect in the initial phase of the pandemic.</p><p><strong>Figure 3: Dynamics in Polarization of House Twitter Accounts</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sv_VHv9naA-F3Xyi" /></figure><p>Second, Figures 4 and 5, showing the discussion space in the California State Legislature suggests that evidence for political division appears stronger at the national level. In the state legislatures, legislators tend to share similar stances on the issues. There are a number of plausible hypotheses that might explain this result. State houses might be less influenced by national media, or state legislators tend to be less attentive to nationally polarizing social issues. Or perhaps these are simply characteristics of state legislative politics in these very blue or very red state legislatures. Clearly more research is needed to determine why we see these differences in political divisions between state and national legislatures.</p><p><strong>Figure 4: Dynamics in Polarization of CA State Senate and Assembly Twitter Accounts</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pk64q4uUXMpDnzY6" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*neDisXgqrl5r-u2G" /></figure><p>We hope these measures will help researchers further study applications to political polarization, political communication strategy, and the dynamics of political messaging.</p><p><strong>About the author.</strong> Daniel Ebanks is a PhD student in social science and computational social science at Caltech. This research was made possible by an internship from Nvidia. I thank Bojan Tunguz (Nvidia), R. Michael Alvarez (Caltech), Betsy Sinclair (WUSTL), and Joon Park (Caltech) for their collaboration on this project.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f21c2b8a98" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/dynamics-of-political-polarization-surprising-stability-in-the-trump-era-congress-f21c2b8a98">Dynamics of Political Polarization: Insights from Using Machine Learning and Natural Language…</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are Canadian Political Parties Responsive to the Climate Change’s Public Salience in Question…]]></title>
            <link>https://medium.com/trustworthy-social-media/are-canadian-political-parties-responsive-to-the-climate-changes-public-salience-in-question-61ab3bfa54a?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/61ab3bfa54a</guid>
            <dc:creator><![CDATA[Jacob Morrier]]></dc:creator>
            <pubDate>Thu, 07 Jul 2022 20:02:23 GMT</pubDate>
            <atom:updated>2022-08-05T22:53:59.303Z</atom:updated>
            <content:encoded><![CDATA[<h3><em>Are Canadian Political Parties Responsive to Climate Change’s Public Salience in Question Period?</em></h3><p>In a <a href="https://preprints.apsanet.org/engage/apsa/article-details/62c51e39f519391c5d7ff254">new working paper</a>, <a href="https://medium.com/u/7e735bbab7be">R. Michael Alvarez</a> and I study issue responsiveness in Canadian politics. Specifically, we consider whether political parties are responsive to their constituents’ policy priorities in choosing the topics they publicly address. We focus our analysis on two of the most pressing policy issues of our time: the environment and climate change.</p><p>We use novel measures of issue attention and salience to conduct our analysis. In particular, we quantify the attention political parties pay to the various issues through the topic composition of the speeches emanating from their members during Question Period and policy issues’ public salience by their popularity on Google’s search engine. We believe that both measures are representative of the attention and importance politicians and members of the public attribute to policy issues.</p><p><a href="https://en.wikipedia.org/wiki/Question_Period">Question Period</a> is a focal moment in Canadian politics. This 45-minute segment occurs every day the House of Commons sits. Its purpose is to offer members of Parliament an opportunity to seek information on the issues of the day and hold the government accountable for its decisions. We employ <a href="https://en.wikipedia.org/wiki/Topic_model">topic models</a> to reduce the dimension of inherently high-dimensional transcripts and uncover their latent topic composition.</p><blockquote>The environment and climate change remain disputed issues, with discernible partisan heterogeneity in attitudes toward them. We expect the latter to result in partisan heterogeneity in issue responsiveness.</blockquote><p>In particular, we expect issue responsiveness to vary along two dimensions. First, since parties do not wish to draw attention to problems over which they have a weak reputation since it would weaken their position, they selectively emphasize the issues over which they have a strong(er) reputation and neglect those over which they have a weak(er) reputation. Accordingly, <strong>we expect the Conservatives to be less responsive to changes in the environment and climate change’s public salience than the Liberals and New Democrats</strong>. Second, through sustained inquiries, opposition parties may compel the government to address policy issues it would otherwise disregard because they are either disadvantageous or embarrassing. The government will ultimately give in to some of the opposition’s pressures and address compromising topics. Accordingly, <strong>we expect the Conservatives to be more responsive to the environment and climate change’s public salience in government than in the opposition</strong>.</p><p>The previous literature has shown that there usually is a strong correlation between the public’s priorities and their political representatives’ agenda.</p><blockquote>The figure below corroborates this stylized fact: <strong>political parties talk more about the environment and climate during the Question Period when these policy issues are more salient to the public</strong>.</blockquote><p>For context, this figure depicts, on the <em>y</em>-axis, the log ratio of a party’s speeches dedicated to the environment and climate change in a given week and, on the <em>x</em>-axis, the log-transformed public salience of these issues in Canada over the same period. The relationship between both variables is essentially linear, and its slope is practically indistinguishable between the parties.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzPweGuHYKuv7BepWTXRuQ.png" /><figcaption><em>Relationship between the Environment and Climate Change’s Prevalence in Question Period and Public Salience</em></figcaption></figure><p>Even though previous results show that there is a positive correlation between policy issues’ salience and the topic composition of parties’ speeches, we cannot affirm at this point that the parties alter their actions <em>in reaction to</em> changes in their constituents’ priorities, let alone provide a causal estimate of parties’ issue responsiveness.</p><p>Two mechanisms can justify the relationship between the public’s policy priorities and political representatives’ attention. On the one hand, politicians have strong incentives to alter their rhetoric to address chiefly the policy issues salient to their constituents. On the other hand, politicians can engage in “public agenda-setting” and influence the weight voters attribute to the various policy dimensions. Reality lies between these two conflicting theories: even if voters’ sense of priorities is susceptible to manipulation, politicians cannot control all the relevant factors, and exogenous events alter policy issues’ salience. It follows that <strong>the relationship between politicians’ attention to policy issues and their public salience is afflicted by simultaneous causality</strong>.</p><blockquote>By naively observing the correlation between the topic composition of politicians’ public statements and issues’ public salience, we might be under the illusion that politicians are being responsive to their constituents when, instead, citizens’ sense of priorities is being manipulated by politicians’ rhetoric.</blockquote><p><strong>We implement a </strong><a href="https://en.wikipedia.org/wiki/Instrumental_variables_estimation"><strong>two-stage least squares (2SLS)</strong></a><strong> estimation strategy to remedy this problem.</strong> In particular, we instrument the environment and climate change’s public salience in Canada with the analogous variable for the United States. This identification strategy allows us to disentangle the effect of public salience on parties’ speeches (i.e., what we want to measure) from the effect of speeches on salience.</p><p>The following table presents estimates of parties’ issue responsiveness. The table contains ordinary least squares (OLS) and 2SLS estimates of four model specifications. These specifications vary along two dimensions: (i) whether partisan heterogeneity in issue responsiveness is permitted or not, and (ii) whether issue responsiveness is allowed to differ after the 2015 election when the Liberal Party replaced the Conservative Party in government.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*doflvz2eXeJn8CCVDMMnLA.png" /><figcaption>Causal Estimates of Issue Responsiveness</figcaption></figure><p>We highlight three key results. First, <strong>all parties are responsive and alter their statements’ topic composition in reaction to exogenous changes in the environment and climate change’s public salience.</strong> As seen in column (5), the ratio of a party’s speeches related to the environment and climate change increases on average by 0.388 % following a one percent increase in these issues’ public salience.</p><blockquote>Second, <strong>two-stage least squares estimation reveals significant heterogeneity in issue responsiveness across the three parties</strong>.</blockquote><p>Unexpectedly, the Liberal Party is <em>less</em> responsive to changes in the environment and climate change’s public salience than the Conservative Party and the NDP. However, this does not imply that the Liberals talk less about the environment and climate change. Indeed, Liberals seem actually to talk about these issues on their terms. This can be seen by comparing columns (2) and (6): estimation via instrumental variables lowers the estimate of the Liberals’ issue responsiveness by 25 %. This suggests that <strong>the Liberals had successfully bolstered the environment and climate change’s public salience over our study period</strong>, especially when they were in government.</p><p>Third, the Conservative Party and the New Democratic Party have the same estimated level of issue responsiveness. This is remarkable because the environment and climate change have historically been weaker policy issues for the Conservatives. The fact that the extent to which they react to changes in the environment and climate change’s public salience is not significantly different from the New Democrats’ gives credence to <strong>the power of Question Period in pushing the party in government to address embarrassing or otherwise unfavorable policy issues.</strong></p><p>In addition to parties’ issue responsiveness, we evaluate whether changes in the environment and climate change’s public salience affect the topic composition of Question Period speeches across other topics<em>. </em><strong>We have found evidence that the Conservative Party engaged in obfuscation in reaction to changes in the environment and climate change’s public salience.</strong> For instance, the Conservatives obfuscate attention paid to climate change by putting forth economic considerations. Besides, following an increase in the environment and climate change’s public salience, the Conservative Party significantly increases the prevalence of intergovernmental affairs in its Question Period speeches. Since federalism is one of the fundamental principles over which the CPC was founded, this result seems a priori consistent with its ideological matrix. That said, it might also indicate a systematic attempt to minimize the federal government’s responsibility in addressing climate change and pass it on to the provinces and territories instead.</p><p>In conclusion, we make notable contributions to the study of political representation and issue responsiveness in Canadian politics. We demonstrated the power of the Question Period in delivering democratic accountability. By attributing agenda-setting power to opposition parties, Question Period allows them to compel the party in government to address issues salient to the public that it may otherwise avoid or neglect. We also uncovered evidence that, during its time in government, the Liberal Party has successfully bolstered the environment and climate change’s public salience through its rhetoric. This result stresses the parties’ role in shaping the political agenda and drawing attention to emerging policy concerns such as the climate crisis. Finally, from a methodological perspective, this paper exemplifies how machine learning can be combined with standard causal identification strategies to answer substantive questions in political science. The methodological innovations we have introduced will considerably benefit the literature on political representation and issue responsiveness.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61ab3bfa54a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/are-canadian-political-parties-responsive-to-the-climate-changes-public-salience-in-question-61ab3bfa54a">Are Canadian Political Parties Responsive to the Climate Change’s Public Salience in Question…</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Following the leader: Congressional Communication on Social Media]]></title>
            <link>https://medium.com/trustworthy-social-media/following-the-leader-congressional-communication-on-social-media-b479675aac4f?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/b479675aac4f</guid>
            <category><![CDATA[party-leadership]]></category>
            <category><![CDATA[strategic-communication]]></category>
            <category><![CDATA[congress]]></category>
            <category><![CDATA[congressional-debate]]></category>
            <category><![CDATA[twitter]]></category>
            <dc:creator><![CDATA[Danny Ebanks]]></dc:creator>
            <pubDate>Wed, 14 Jul 2021 16:11:13 GMT</pubDate>
            <atom:updated>2021-07-14T16:11:13.431Z</atom:updated>
            <content:encoded><![CDATA[<p>Over the last decade, social media has become a key platform for political leaders to communicate with the public; however, political elites also use these platforms to strategically communicate with one another. <a href="https://preprints.apsanet.org/engage/apsa/article-details/60c239b28214c646e0a61589">In a recent paper, Hao Yan, Mike Alvarez, Sanmay Das, Betsy Sinclair, and I find that patterns of communication on social media follow predictable patterns from a game theoretic model of message coordination among the party leadership and their rank-and-file members.</a> These findings hint at the possibility of using social media to understand the strategic interactions of members of Congress.</p><p>Our results suggest that leadership’s ability to influence the terms of discussions depends on balancing two key forces: coordinating around a policy stance and successfully aggregating information where policy outcomes are uncertain. Party leaders and rank-and-file must balance coordinating around any position (the coordination problem), but also selecting the correct position (the information problem).</p><p>Depending on which problem dominates, party rank-and-file will lead or follow with respect to their strategic communication strategy. When the party has a low need for policy direction, the coordination problem dominates. So, the rank-and-file turns toward the leadership whose messaging acts as a rallying point around which to coordinate. Thus, we expect that topics where the need for policy direction is low will be the topics where House leaders initiate discussion for the rank-and-file. By high need for direction, we mean that the electoral, political, or policy costs of coordinating on the “wrong’’ policy stance are large. In the case of coordinating on the wrong stance, when need for direction is high, it might be more advantageous to avoid coordination. For example, coordinating on the wrong stance could exacerbate a government shutdown crisis, leading to electoral defeat for the party. By low need for direction, we mean that the effects (good or bad) for coordinating on the “right’’ or “wrong’’ policy stance are not large — perhaps only a positive or negative cycle of news coverage.</p><p>To find topics where need for direction is low, we follow the subsequent procedure: we identify topics topics that drive separation in between the two parties (for example, immigration), we expect the information problem is least acute. For the most polarizing topics, this is precisely because these are policy stances which define belonging to a particular party. Thus, in theory, it is on these types of partisan topics that leaders have the most influence over the rank-and-file, since the outsized costs or benefits of coordinating on the wrong messaging are low. Here, the coordination problem should predominate, and we expect party leaders to take the initiative in defining the messaging.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2BV1vM_a9XZ4YsueObtgSg.jpeg" /><figcaption><strong>Figure 1: </strong>Impulse Response Functions for sentiment-topics predicted to be leader driven for the Democratic Party. Bootstrapped 95-percent confidence intervals are shown. Asterisks indicate sentiment-topics where a Granger test was statistically significant for leaders on rank-and-file.</figcaption></figure><p>We show results from each party above. Specifically, Figure 1 shows that Democratic House leaders exerted the most influence over the propensity to discuss Trump immigration policies (approximately a 1 percentage point increase for each standard deviation shock) and protecting health care (a ~0.5 percentage point increase). House leaders exerted a ~0.5 percentage point increase for gun policy, presidential power, combating climate change, child separations, and women’s reproductive health. On these same topics, Figure 1 shows that House rank-and-file exerted a 4 percentage point increase on the leader’s propensity to discuss President Trump’s immigration policies, a 3 percentage point increase for protecting health care, and 2 percentage point increases for gun policy, presidential power, combating climate change, child separations, and women’s reproductive health. Notably, these effect sizes are an order of magnitude higher than the leadership’s influence on rank-and-file members.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0PiV62Plhg6UUGnGttwX-Q.jpeg" /><figcaption><strong>Figure 2: </strong>Impulse Response Functions for sentiment-topics predicted to be leader driven for the Republican Party. Bootstrapped 95-percent confidence intervals are shown. Asterisks indicate sentiment-topics where a Granger test was statistically significant for leaders on rank-and-file.</figcaption></figure><p>The Republican party exhibits behavior consistent with the Democratic party. Figure 2 shows that impulses of ~10 percent to the GOP House leaders’ daily propensity to discuss a particular issue results in a less than 1 percent increase in the rank-and-file members’ daily propensity to discuss that issue. In particular, Republican leaders induced a 1 percentage point increase in their rank-and-file members’ propensity to discuss impeachment. Leaders induced a 0.5 percentage point increase for trade deals and USMCA. For protecting health care, equality for women, jobs and the economy, presidential power, and agricultural policy, GOP House leadership exerted a 0.2 percentage point increase on the rank-and-file members’ propensities to discuss these sentiment-topics. Figure 2 also shows that members induced a 3.5 percentage point increase in their leadership’s propensity to discuss impeachment, and a 1.5 percentage point increase for trade deals and USMCA. For protecting health care, equality for women, jobs and the economy, presidential power, and agricultural policy, GOP leadership exerted a 0.5 percentage point increase on the rank-and-file members’ propensities to discuss these sentiment-topics. Again, members’ influence is an order of magnitude larger than the leadership’s influence. Notable, the magnitudes derived for Republicans leadership and rank-and-file members are smaller than for Democratic leaders and members. This suggests that Democratic party leaders and members are more responsive to each other with respect to their messaging around their propensity to discuss sentiment-topics.</p><p>This evidence from the topical analysis suggests that leaders exert influence over their rank-and-file members on topics that come to dominate social media discussion, and vice versa. Importantly, although it is clear House leadership and rank-and-file messaging on Twitter influence each other, we find consistent with game theoretic frameworks of Congressional strategic communication, leaders tend to initiate discussion and messaging on topics where the need for policy direction is low and the coordination problem predominates. Interestingly, rank-and-file members are able to exert nearly double the influence over their leaders as in the reverse. This suggests the power of leaders to drive messaging is not necessarily absolute in the US House, even on a highly scripted platform such as Twitter.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b479675aac4f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/following-the-leader-congressional-communication-on-social-media-b479675aac4f">Following the leader: Congressional Communication on Social Media</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2021 Inauguration on Twitter]]></title>
            <link>https://medium.com/trustworthy-social-media/2021-inauguration-on-twitter-14aa6d37cc17?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/14aa6d37cc17</guid>
            <category><![CDATA[twitter]]></category>
            <category><![CDATA[inauguration]]></category>
            <category><![CDATA[online-harassment]]></category>
            <category><![CDATA[social-media]]></category>
            <dc:creator><![CDATA[Maya Srikanth]]></dc:creator>
            <pubDate>Mon, 08 Mar 2021 16:42:20 GMT</pubDate>
            <atom:updated>2021-03-08T16:42:20.349Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>We recently tested our <a href="https://medium.com/trustworthy-social-media/finding-social-media-trolls-with-dynamic-keyword-selection-b6754c8aefef">dynamic data collection method</a> using word embeddings and corpus frequencies to track Twitter discussions about the 2021 presidential inauguration. In this post, we cover the visualizations and metrics used throughout our mini study, a browser-based UI used for human-assisted data collection, and some qualitative findings.</p><h3>Data Visualization Platform</h3><p>While our dynamic method can theoretically be fully automated, we are currently testing a human-in-the-loop process for keyword selection.</p><p>To that end, we are building a data visualization platform which processes new twitter data, runs machine learning models on this data, and dynamically updates frontend browser-based UI to reflect changes in live social media discussions and recommend new keywords for data collection. The figure below shows the full cycle of the backend and frontend of our platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*19ok5uhRTTEg0ORtMFxAzA.png" /><figcaption>Workflow of our Data Collection, Storage, Analysis, and Visualization Platform. Here we use Twitter APIs as an example for data collection and we use Google Cloud Platform (GCP) as an example for data streaming and storage. GCP Monitor works in a sequential way while the backend analyses can be parallel.</figcaption></figure><p>Below are some key UI elements on this platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9iKs2kmo7xiS3K2VKibKww.png" /><figcaption>Forecast plot generated after training ARIMA on log-transformed #insurrection frequency data, which predicts log frequencies 15 timesteps into future and provides a 95% confidence interval.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fif7EDs658BuyPhRXpiGaA.png" /><figcaption>Interactive tsne (t-distributed stochastic neighbor embedding) plot of the closest 30 neighbors to the tracked keyword #insurrection (shown in blue).</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*fWc76ydmy3GlqA27_pF3iA.png" /><figcaption>Table of 30 closest neighbors to #insurrection, sorted by a linear combination of keyword cosine distance to #insurrection and corpus frequency</figcaption></figure><p>We used a prototype of this platform to make human-assisted keyword selections throughout the 2021 inauguration case study.</p><h3><strong>Experiment Details</strong></h3><p>We started our dynamic twitter monitor on 2021 on January 11, 2021 (at 10:40:44 Pacific Standard Time), with a single keyword, “inauguration.” From then on until January 22, 2021, we used our dynamic keyword selection method to choose new keywords for data collection using Twitter’s streaming API.</p><p>The figure below shows the hourly volume of inauguration-related tweets detected by our dynamic monitor throughout the experiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f_Uwu_krU3cLamMTyu0_lg.png" /><figcaption>Fig 1: Volume of tweets collected by a dynamic monitor based on word embeddings and frequencies and static monitor using only 1 keyword (“inauguration”) to collect data.</figcaption></figure><p>Each day of the experiment, we used a prototype of our data visualization platform to visualize important metrics (corpus frequencies, GloVe embeddings, nearest neighbors) and select new keywords for data collection. Our browser-based UI allowed our research group to streamline a human-assisted keyword selection process.</p><h3>Observations</h3><p>From the word clouds below, we see that our dynamic monitor covers a wider range of topics and has a more uniform distribution in terms of the frequency of different keywords when compared to a static monitor (which uses only “inauguration” to pull data).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*aAiTLT8TR5OMz9sh9PWBfg.png" /><figcaption>Word cloud from the <strong>static monitor</strong> (left) and the <strong>dynamic monitor</strong> (right) on day 9 of the experiment showing the most frequently used hashtags in the statically and dynamically obtained Twitter datasets. Size of word corresponds to frequency of word within corpus.</figcaption></figure><p>Our dynamic method revealed trending terms which cropped up from Jan 11 to Jan 22, like the high-frequency keywords shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*wHf6fuK51QWHXj_kwNbVpA.png" /></figure><p>Further, our data visualization platform allowed us to clearly trace the evolution of our dynamic keyword set throughout the experiment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uEEqBxHHQ6thwTAn_UAsOQ.png" /><figcaption>Keywords update (add and remove) by the dynamic monitor for data collection in real-time case study of 2021 #inauguration discussions on Twitter.</figcaption></figure><p>Let’s provide some real-world context to the keywords in the figure above. Emergence of ‘#nationalguard’ on Jan 12 likely corresponds with the 2021 capitol protests (as does ‘#capitolriot’ and ‘#capitolbuilding’ on Jan 16), and emergence of ‘databreach’ on Jan 13 is likely linked to the hacking of Parler’s data. As time goes on, we see that the monitor begins to focus on politicians, detecting various forms of ‘#biden’ (‘#joebiden’, ‘#bidenharris’ ‘#bideninauguration’, ‘#bidenharrisinauguration’), ‘#trump’ (‘#donaldtrump’), and ‘#kamalaharris’. After inauguration on Jan 20th, all #inauguration discussions naturally lost traction (see Figure 10 for declining frequencies after the fact).</p><p>In general, we find that using our dynamic keyword selection method to generate keyword recommendations and graphics on our data visualization platform made for a more informative and thorough data collection process. For more details on case study findings, experiment design, and the data visualization platform, see our <a href="https://arxiv.org/pdf/2102.12596.pdf">recent arxiv submission</a>.</p><h3>Next Steps</h3><p>As an immediate next step, we plan to incorporate predictive time series modeling into our algorithm as another datapoint for keyword selection. Further, in the next few months, we aim to publicly release code for the data visualization platform so others can try this method out on their own twitter data. Thanks for reading, and stay tuned for more work!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14aa6d37cc17" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/2021-inauguration-on-twitter-14aa6d37cc17">2021 Inauguration on Twitter</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Thoughts on Challenges in Computational Social Science]]></title>
            <link>https://medium.com/trustworthy-social-media/thoughts-on-challenges-in-computational-social-science-eb0514d5cf3f?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/eb0514d5cf3f</guid>
            <category><![CDATA[interdisciplinary]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Anqi Liu]]></dc:creator>
            <pubDate>Tue, 19 Jan 2021 23:16:06 GMT</pubDate>
            <atom:updated>2021-01-19T23:16:05.935Z</atom:updated>
            <content:encoded><![CDATA[<h3>Four Challenges for Computational Social Science</h3><p>It is cool to do interdisciplinary research, but what does it take? Computational Social Science as an example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*w2AEN4dQXSJocC4sib4iFA.png" /></figure><h3><strong>My motivation for this post:</strong></h3><p>During my postdoc at Caltech, I started to utilize machine learning and artificial intelligence (ML/AI) to study social science problems. I have enjoyed the opportunity to talk to students and senior researchers in social science who share the same interest. Coming from a computer science background, it is quite an education for me to align my language with experts in a different field and look at problems differently. For example, even though people in ML get concerned about issues like echo chamber effects, confirmation bias, hate speech, and social media trolling, conducting research on mitigating them takes more than our usual routine. Here are some of my thoughts on the challenges we face. The first two are about specific difficulties, and the last two are more generally about building research relations and the research process.</p><h3><strong>Challenge 1: How to get the data and the problem of domain shift.</strong></h3><p>In social media analysis, we used to collect our data by using platform APIs and scraping websites. It may be fine if our problem is, for example, link prediction for generating better friend recommendations. However, there are many completeness and efficiency limitations in this approach when it comes to monitoring fast-evolving discussions and studying moderations/management of platforms. <a href="https://arxiv.org/abs/2005.02442v2">This paper</a> describes the challenge in more detail. Surprisingly, there was no good solution for an efficient and reliable data collection mechanism for real-time social media data before. We developed such a method by ourselves, which benefits many of <a href="https://anqiliu-ai.github.io/project/social/">our projects</a>.</p><p>Another issue is there are large domain shifts between different time periods or different discussion spaces. Machine learning usually depends on the assumption that the data distribution in our training data is the same as the one in the testing. However, domain or distribution shift is everywhere. In many problems, it is possible that historical data is not representative enough due to difficulties in data collection or just the nature of the problem. This prevents us from using data from one domain and generalizing our conclusion to the other one. For example, at the beginning of the #MeToo movement, the frequency of the keyword “MeToo” increases so fast that no model can capture this trend using historical data. It also reflects the importance of having access to the real data instead of relying on synthetic or historical data in our research. Domain shift is a fundamental problem in the machine learning area. Check my related projects <a href="https://anqiliu-ai.github.io/project/drl/">here</a>.</p><h3><strong>Challenge 2: The “ground truth” problem, and what is the evaluation criterion?</strong></h3><p>Coming from an ML background, I am so used to having access to “ground truth,” whether it is the image label in image classification or the expert policy in imitation learning. I am used to having clear evaluation criteria, such as accuracy, <a href="https://en.wikipedia.org/wiki/F-score">F-1 score</a>, or <a href="https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Precision_at_K">precision@k</a>. In Computational Social Science, even though we also need to evaluate how good the model is, the criterion and ground truth are not as obvious and need to be chosen carefully. For example, if we want to create Trustworthy Social Media, instead of having A/B testing on different policies for longer user engagement, what criterion should we use to determine whether we have achieved our goal? Is it the diversity of opinions, the users’ satisfaction level by a survey, or the overall sentiment? When <a href="https://anqiliu-ai.github.io/files/social_projects/wiml.png">modeling topic evolution using real social media data</a>, how should we do <a href="https://en.wikipedia.org/wiki/Model_selection">model selection</a>? Should we depend more on our common sense or the quantatitive measures like <a href="https://en.wikipedia.org/wiki/Perplexity">perplexity</a>? You may have figured out that there are objective criteria and subjective ones. We may need to transfer subjective criteria to objective numbers for the quantitative analysis. But taking into human perspectives would be a must if we go back to the core problem we care about here: creating Trustworthy Social Media to ensure a better online environment for everyone.</p><h3><strong>Challenge 3: Too many assumptions: What we think social scientists need and what they really need.</strong></h3><p>Before my collaboration with social scientists, I had many assumptions about the kind of research they are doing. I assumed they care more about causality rather than predictability. I assumed they want to interpret the model more than using the model. I thought they only use “small data” from human subject experiments. This may be true for some researchers, but it really depends on the actual problems they are working on. For our projects on dynamic keyword search, predicting the trending words for the next period helps us improve our monitors for data collection. We have a huge social media dataset to deal with, even though we will rely on human subject experiments to testify some interventional strategies. There is also a strong need for accurate uncertainty quantification methods in many predictive problems. Therefore, if you are interested in social science questions but have assumptions that prevent you from talking to domain experts, it’s time to throw away the biased assumptions and start the conversation.</p><h3><strong>Challenge 4: Hard to make a change: We should join forces!</strong></h3><p>Making the (social media) world a better place is a very complicated problem. There are researchers from both the social science world and the ML/AI/engineering world who want to commit to this. But we seem to be missing some essential components in our research to really make a change.</p><p>From the social science point of view, most of the research in Computational Social Science focuses on analyzing the software engineering outcome directly. For example, we tell the public how social media platforms facilitate polarization and how it may affect our beliefs and preferences in a way that we are not aware of. But these platforms are based on algorithms and software infrastructure optimizing unknown objectives. Rethinking from the beginning of the pipeline and studying the design of the platform would be necessary. This may require help from the engineering field.</p><p>From a pure engineering perspective, the way we think about our research may need to be changed. Previously, researchers conduct fundamental research and publish the results without worrying about actual downstream applications. However, in a software engineering pipeline, if the researchers do not note the potential negative consequences if the method is misused, the engineering and commercial sectors may not either. (This is very much inspired by <a href="https://nips.cc/virtual/2020/public/invited_16166.html">Charles Isbell’s</a> and <a href="https://www.microsoft.com/en-us/research/people/wallach/">Hanna Wallach</a>’s talk at the last Neurips conference.) To mitigate the problem from the beginning, we need help from the social science field to pick up useful approaches to think about our ML/AI research.</p><p>Therefore, develop Trustworthy Social Media, should join forces!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb0514d5cf3f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/thoughts-on-challenges-in-computational-social-science-eb0514d5cf3f">Thoughts on Challenges in Computational Social Science</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Five Important Social Media Trends in 2021]]></title>
            <link>https://medium.com/trustworthy-social-media/five-important-social-media-trends-in-2021-78862e71d51d?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/78862e71d51d</guid>
            <category><![CDATA[facebook]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[2021-trends]]></category>
            <category><![CDATA[twitter]]></category>
            <dc:creator><![CDATA[R. Michael Alvarez]]></dc:creator>
            <pubDate>Wed, 30 Dec 2020 19:33:42 GMT</pubDate>
            <atom:updated>2020-12-30T19:33:42.546Z</atom:updated>
            <content:encoded><![CDATA[<p>What are the five important social media trends that the research community needs to pay attention to in 2021?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*lBYVUoqXY_FVjy1Y7cblYA.jpeg" /></figure><h3>1) New Regulatory and Legal Actions.</h3><p>We’ve all heard <a href="https://www.cnn.com/2020/12/10/tech/facebook-antitrust-lawsuit-analysis/index.html">the buzz about breaking up Facebook</a>, and calls for more regulation of social media and online sales giants like Google and Amazon. There’s no doubt that in the United States, with a new Democratic presidential administration taking office in late January, that we’ll hear more about regulating social media platforms and generating a more competitive social media environment.</p><p>While I’m not thinking that there will be big changes in the regulatory environment for social media platforms in 2021, with lawsuits and scrutiny of their business practices, I think that Google, Facebook, Twitter, and Amazon might engage in some (needed) corporate self-reflection, and that they might themselves take some steps to alleviate some of the political pressure on their business models.</p><p>One change in 2021 might be that social media platforms, Facebook in particular, will be more open and transparent about their platforms, and that as part of that movement, will provide researchers with improved access to data from their platform (<a href="https://medium.com/trustworthy-social-media/misinformation-on-facebook-aacada0d2f71">something that I wrote about recently in another Trustworthy Social Media post</a>). Twitter will be taking a number of steps in 2021 to give academics and researchers improved access to data from their platform, and that also might put some pressure on the other social media and online commerce giants to do the same.</p><p>I also suspect that we will see more efforts to push the social media giants to better police the use of their platforms. For example, doing a better and more transparent job of detecting misinformation, eliminating trolling and harassment, and generally making the use of their platforms a positive and informative experience, may improve in 2021. Providing users and researchers with more information about how they are enforcing their policies might improve in 2021. Researchers will want to stay well-informed about these efforts, and about <a href="https://medium.com/trustworthy-social-media/finding-social-media-trolls-with-dynamic-keyword-selection-b6754c8aefef">research that is being done to detect trolling, harassment, and misinformation online.</a></p><p>Finally, I wouldn’t be surprised to see some of the states take the lead in pushing for more regulation of social media platforms, in particular states (like California) that have the initiative and referendum process. <a href="https://www.jstor.org/stable/42864374">Political science researchers have shown that strategic politicians use the initiative and referendum process to put issues like these in front of voters,</a> and I would not be surprised if we hear discussion of potential ballot measures to further regulate social media platforms in 2021.</p><h3>2) Increasing Use of Video Content.</h3><p>It doesn’t take a Ph.D. researcher to tell you that social media platforms are seeing more and more video content (and that they are giving their users improved tools to post and edit video content). While the research community has made great advances in the analysis of the text information generated by social media platforms, we still lag behind the user community in our tools for collecting, storing, and analyzing the vast amount of video content that is being shared today on social media platforms.</p><p>By not analyzing video social media content, the research community is really missing out on a lot of important information. A social media post might have a short snippet of text, with an accompanying video, where the text does not adequately describe the details of the content in the video. Or that text snippet might be phrased in a way that leads natural-language processing methods astray, and we actually don’t get the correct interpretation of the content because we ignore the video.</p><p>The good news is that researchers are actively working to develop tools and methods that allow for the quick and easy analysis of images and videos; an excellent introduction is in this book, <a href="https://www.cambridge.org/core/elements/images-as-data-for-social-science-research/0376EE8A7A21F5B47FC4EC24DF07EFE9">Images as Data for Social Science Research</a> (recently published in a book series that I co-edit). While our methods for analysis of video social media content are lagging the dissemination of video content, I am sure that in 2021 we will see an increased focus on the development of tools for analysis of video content by researchers. I also anticipate that we see an increase in the quantity of interesting research papers published in 2021 that use data from social media video content.</p><h3>3) Now You See It, Now You Don’t.</h3><p>One issue that might be troubling for researchers is the trend towards the increased use of so-called “ephemeral content.” This is social media content that is posted but which automatically disappears after some period of time, and it’s becoming increasingly popular (for example, Snapchat and Instagram).</p><p>In 2021, we will see social media platforms provide their users with more tools to post transitory content. And I suspect that we will also see that users will take advantage of these tools. These trends pose problems for many types of research that are conducted using social media data, especially those that collect or scrape social media posts days, weeks, or months after they were originally published.</p><p>If more ephemeral content is posted in 2021, researchers might increasingly be missing out on social media platform conversations. If users take advantage of these tools for posting content that quickly disappears, and use them for certain types of conversations, this might introduce different types of biases in social media datasets and analyses. Researchers will need to follow this trend carefully, and in some cases, rethink their data collection methods. And researchers might need to start to think about how to detect and correct for these types of biases in their analyses of datasets where some content might have been self-censored.</p><h3>4) New Social Media Platforms?</h3><p>Don’t get me wrong, Facebook, Twitter, Instagram, and the other big social media platforms won’t disappear in 2021. But I think that we will continue to see new social media platforms emerge in 2021 (after all, I don’t think that anyone saw that TikTok would become so popular, and so controversial, in 2020!).</p><p>While new social media platforms won’t replace their bigger rivals in 2021, I think that we will see the continued emergence of new social media platforms in 2021. Researchers should pay careful attention to these new entrants, because like TikTok they will likely focus on trying to gain user shares quickly among very specific demographics, with very specific types of applications. If many members of specific demographic groups suddenly shift their social media from Facebook or Twitter to a new platform, that again produces potential biases in our data. In 2021 researchers pay more attention to these new social media platforms, like TikTok, which haven’t received a great deal of attention in the research literature.</p><h3>5) New Tools for Analysis of Social Media Data.</h3><p>Finally, I’m excited and confident that we will see many new tools and methods in 2021 for the analysis of social media. The research community has made great advances in the development of tools and methods for analyzing the text in social media data, and we will see the continuation of this in 2021. I think we will see text and natural language processing tools that will be faster and that will work with data at larger scales; our Trustworthy Social Media group is working on some new topic models methods in very large scale text datasets, for example.</p><p>Also, as I discussed earlier, there’s going to be a lot more development of tools for the collection and analysis of images and video. With so much social media content today including images and video, researchers will need to analyze those content alongside the accompanying text, which will complicate the computational tasks of working with social media data.</p><p>Finally, I hope that in 2021 we also see that researchers develop and release more tools for the collection of social media data. <a href="https://medium.com/trustworthy-social-media/how-do-we-monitor-social-media-bdd6e9dc6c3c">Our group has released the code for our cloud-based social media collection infrastructure</a>, and I expect that we will see other research groups share similar code and tools in 2021.</p><h3>Conclusion</h3><p>In conclusion, 2021 is going to be a challenging and exciting time to be collecting and analyzing social media data. With so much of the world’s communications shifting to social media platforms, with the emergence of new social media tools, and with the shift towards video content, we will have a lot of work in 2021 to continue our research towards building more Trustworthy Social Media. I’m confident that 2021 is going to see a great deal of innovating and exciting research and scientific developments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78862e71d51d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/five-important-social-media-trends-in-2021-78862e71d51d">Five Important Social Media Trends in 2021</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Misinformation on Facebook]]></title>
            <link>https://medium.com/trustworthy-social-media/misinformation-on-facebook-aacada0d2f71?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/aacada0d2f71</guid>
            <category><![CDATA[facebook]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[misinformation]]></category>
            <category><![CDATA[twitter]]></category>
            <dc:creator><![CDATA[R. Michael Alvarez]]></dc:creator>
            <pubDate>Tue, 29 Dec 2020 18:08:54 GMT</pubDate>
            <atom:updated>2020-12-29T18:08:53.953Z</atom:updated>
            <content:encoded><![CDATA[<p>As researchers, we do not know enough about misinformation on Facebook.</p><p>Why? Because Facebook does not give researchers easy access to content from their platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*qCKfJWn-gcO8Ba8gLbbpxw.jpeg" /></figure><p>Everyone hears stories about the rampant misinformation and negativity on Facebook. Recently, Charlie Warzel wrote an interesting article about this trend in the New York Times titled <a href="https://www.nytimes.com/2020/11/24/opinion/facebook-disinformation-boomers.html?searchResultPosition=3">“What Facebook Fed the Baby Boomers.”</a> In the article, he discusses how he worked with two baby boomers’ to access their Facebook feeds, and describes just how much toxic and misleading content the subjects were exposed to. Warzel summarizes the experience of one of the two baby boomers as “an information hellscape.”</p><p>It’s an interesting read, but a question remains: why did Warzel take the approach of following just two users? He writes in the article:</p><blockquote>“Despite Facebook’s reputation as a leading source for conspiracy theories and misinformation, what goes on in most average Americans’ news feeds is nearly impossible for outsiders to observe. Tools like CrowdTangle, which track “engagements” with social media posts, are the best available means to understand what is popular on the platform, though Facebook (which owns the CrowdTangle) argues that CrowdTangle is not a reliable indicator for how many people saw a post.”</blockquote><p>Unfortunately, the approach used by Warzel is one of the few ways that a researcher can get access to the sort of information that flows through most Facebook accounts. But while Warzel’s approach is interesting (and makes for a great read), it does not provide timely and systematic data that researchers need to understand how much misinformation, harassment, trolling, and hate speech exists on Facebook — nor does this approach give us the data to build tools that can detect and perhaps prevent misinformation on Facebook.</p><p>Of course, you might be thinking, can’t a researcher build a scraper that could quickly collect a great deal of Facebook information? The answer is yes, but that could violate the current <a href="https://www.facebook.com/legal/terms">Facebook Terms of Service</a>. The TOS states: “You many not access or collect data from our Products using automated means (without our prior permission) or attempt to access data you do not have permission to access.” So an effort by an independent researcher to scrape data at scale seems out of the question.</p><p>Of course, many of us know the story that led to these restrictions on data sharing with researchers. In the wake of the Cambridge Analytica scandal (the details of which have been recounted numerous times, for a good overview I’d suggest reading <a href="https://www.amazon.com/Mindf-Cambridge-Analytica-Break-America/dp/1984854631/ref=sr_1_1?dchild=1&amp;keywords=christopher+wylie&amp;qid=1609265191&amp;s=books&amp;sr=1-1">Christopher Wylie’s book Mindf*ck</a>), Facebook essentially shut down research access to data from their platform. In the two and a half years since then, the research community has had very limited access to large, timely, and researcher-oriented datasets from Facebook. Facebook essentially cut off all direct research access to content from their platform.</p><p>Today, Facebook makes available a limited selection of curated datasets for academic research on their <a href="https://research.fb.com/data/">Facebook Data for Independent Research</a> site, but that’s a far cry from allowing researchers to collect their own datasets. There’s also the <a href="https://socialscience.one/">Social Science One</a> initiative, but it involves an opaque process to request access to curated datasets (right now it says that one can apply for access by sending a request on the proposals section of a website that doesn’t seem to exist at this time. I’m happy to be corrected about that if I’m wrong).</p><p>But this is not enough. Facebook has billions of users worldwide, and stories like Warzel’s imply that Facebook may have become one of the primary mechanisms for misinformation and vitriol to spread online. Facebook needs to start restoring direct research access to their platform, following the lead of their competitor, Twitter. Not only has Twitter given researchers direct access to content from their platform, <a href="https://developer.twitter.com/en/products/twitter-api/early-access/guide">but they have also recently announced a series of steps they will take in 2021 to provide qualified researchers even greater access to content from their platform.</a> And by this I mean opening an API that allows research queries for current Facebook content, at the sort of scale that would allow researchers to study the flow of information (and misinformation) on Facebook.</p><p>What are we missing by not having direct and unimpeded access to current content on Facebook? As one example, if our group wanted to analyze the spread of misinformation in the 2020 presidential election on Facebook, in the same way that tracked on monitored these issues using Twitter data, we simply could not replicate this methodology using the Facebook platform. <a href="https://medium.com/trustworthy-social-media/what-are-people-tweeting-about-the-2020-presidential-election-f3dc96ff838b">The word cloud below comes from our Twitter election monitor</a>, and it shows the type of data that we collect from Twitter (which isn’t readily available from Facebook). Nor can we dig deeply into datasets on the 2020 presidential election to detect what may be small but influential networks that spread misinformation on Facebook. Thus we really cannot understand the full ecosystem of misinformation online, because we can’t easily and independently get a complete dataset on what is happening on Facebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*1Cq7AkvyXNdGo8sQMja6MQ.jpeg" /><figcaption>From the Twitter election monitor at monitoringtheelection.us</figcaption></figure><p>There’s even the chance that since those who deliberately spread falsehood online know that the research community has little ability to track their misinformation campaigns on Facebook, they might be more inclined to use Facebook for their activities.</p><p>In order for research groups like ours to help companies like Facebook build a truly <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a>, we need timely and detailed access to data from their platform. We hope that in 2021, Facebook provides better opportunities for researchers to collect the data they need for their studies, to gather data at the scale necessary for in-depth analysis of content and spread of misinformation, and to be able to gather the data without Facebook being able to restrict the types of research that use content from their platform.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aacada0d2f71" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/misinformation-on-facebook-aacada0d2f71">Misinformation on Facebook</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Was YouTube Too Slow To Remove Election Fraud Content?]]></title>
            <link>https://medium.com/trustworthy-social-media/thanks-youtube-but-you-might-be-too-late-7fe559aff069?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/7fe559aff069</guid>
            <category><![CDATA[election-fraud]]></category>
            <category><![CDATA[youtube]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[misinformation]]></category>
            <category><![CDATA[election-2020]]></category>
            <dc:creator><![CDATA[R. Michael Alvarez]]></dc:creator>
            <pubDate>Thu, 10 Dec 2020 22:03:01 GMT</pubDate>
            <atom:updated>2020-12-13T16:24:44.492Z</atom:updated>
            <content:encoded><![CDATA[<p>Yesterday, <a href="https://blog.youtube/news-and-events/supporting-the-2020-us-election/">on their official blog</a>, YouTube announced that they will (now) begin to take down “any piece of content uploaded today (or anytime after) that misleads people by alleging that widespread fraud or errors changed the outcome of the 2020 U.S. presidential election …”. This is a good step, but there’s a strong chance that by not acting earlier to take this content down, YouTube (and other social media platforms) might have fueled voter distrust and lack of confidence, especially among Trump voters and Republicans.</p><p>YouTube justified this change in policy by noting that the safe harbor deadline for the election had passed, and that a sufficiently large number of states have now certified their presidential election results. Whether in YouTube’s opinion that means that the claims of election fraud or errors are now irrelevant, or if they believe the fraud claims do not have factual support, isn’t clear from their post. And while the post notes many of the proactive steps that YouTube took during the election (they mention deleting over 8,000 channels and “thousands of harmful and misleading election-related videos”) are important, that they still allowed misleading content about election fraud with little factual basis to remain on their platform might have lasting consequences for how confident American voters are in the election process. Finally, there are important distinctions between YouTube’s new policy, and the policies that were implemented during the 2020 Presidential election by Facebook and Twitter, but that’s really a topic best taken up in a later story that can really dig into those differences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D16I_Dnx7nhPN-S_WIfgAA.jpeg" /></figure><p><a href="https://www.nadamscohen.com/">Nicholas Adams-Cohen</a>, a member of our research group, wrote a fascinating piece on Medium last week, <a href="https://medium.com/trustworthy-social-media/suppression-or-manipulation-8ad54f4def22">“Suppression or Manipulation? Comparing Perspectives on US Election Security Between Biden and Trump Voters.”</a> He shows a really important figure in his piece, demonstrating the highly partisan split in the type of concerning information received on social media about election integrity during the election. In particular, Nicholas showed that Trump voters were much more likely than Biden voters to receive concerning social media information on voter fraud, voting machine tampering, duplicate voting, and non-citizen voting, many of the same themes that are no doubt in the content that YouTube will now not allow on its platform.</p><p>In a related report, the <a href="https://monitoringtheelection.us/">Monitoring the Election project</a> at Caltech released a <a href="https://static1.squarespace.com/static/5ace8a6b45776eba2e40cbee/t/5fb6d361f0ceb67888b03c1c/1605817186684/Voter_Confidence_2020_Survey_Report.pdf">research memo last week presenting new data on voter confidence in the immediate aftermath of the November 2020 election</a>. This report shows that there are very substantial gaps in voter confidence, in particular regarding whether the election was administered well nationally, between Republican and Democratic registered voters, and also between Trump and Biden supporters.</p><p>Remarkably, the <a href="https://monitoringtheelection.us/">Monitoring the Election</a> report showed that 44% of Trump voters said that they were not at all confident in election administration at the national level, and that another 26% of Trump voters were not too confident. But 48% of Biden voters were very confident in the nation’s election administration, and 39% were somewhat confident.</p><p>Correlation is not causation, and we need to be careful attributing cause-and-effect. But these data imply that since Trump supporters were more exposed to concerning social media claims about voter fraud, and since Trump voters lack confidence in the national administration of the election, that by not doing a better job removing misleading or false claims about voter fraud social media platforms like YouTube might have allowed the distribution of information that has fueled this large partisan gap in voter confidence.</p><p>Clearly we need to do a great deal of additional research to better connect social media content, exposure to that content, and attitudes and perceptions about the integrity of the 2020 presidential election. We need to give social media platforms better guidance, based on the science, about how their content may lead to problematic outcomes that could have lasting consequences for how much trust American voters have in their election process and voting infrastructure. Research groups like ours also need to do more analysis of the effects of these policies, stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7fe559aff069" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/thanks-youtube-but-you-might-be-too-late-7fe559aff069">Was YouTube Too Slow To Remove Election Fraud Content?</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding Social Media Trolls with Dynamic Keyword Selection]]></title>
            <link>https://medium.com/trustworthy-social-media/finding-social-media-trolls-with-dynamic-keyword-selection-b6754c8aefef?source=rss----ace6c515866f---4</link>
            <guid isPermaLink="false">https://medium.com/p/b6754c8aefef</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[trolling]]></category>
            <category><![CDATA[word-embeddings]]></category>
            <category><![CDATA[data-collection]]></category>
            <category><![CDATA[social-media]]></category>
            <dc:creator><![CDATA[Maya Srikanth]]></dc:creator>
            <pubDate>Thu, 03 Dec 2020 16:01:05 GMT</pubDate>
            <atom:updated>2020-12-03T16:01:04.177Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>Introduction</strong></h4><p>Many of us have come to know firsthand that social media is a hub for connection, entertainment, and information… but also a vehicle for online harassment. A 2017 <a href="https://www.pewresearch.org/internet/2017/07/11/online-harassment-2017/">Pew Survey</a> shows that 41% of Americans have experienced online harassment, and 66% have witnessed it. This figure is only on the rise as social media platforms gain popularity.</p><p>Online trolling is difficult to address in part because of the dynamism of social media interactions. Oftentimes, abuse goes undetected due to natural topic evolution in a fast-paced discussion, as well as explicit manipulation of abusive terms by online trolls.</p><p>In 2019, we set out to understand patterns of discussion and harassment in social media data. As a first step in combating online abuse, we aimed to create a social media data collection algorithm that could capture fast topic evolution in general discussion and detect permutations of abusive keywords. In our preliminary analyses, we identified a few AI-driven tools to use for our application:</p><p><strong>1. Closest neighbors:</strong></p><ul><li>GloVe embeddings: training <a href="https://nlp.stanford.edu/projects/glove/">GloVe </a>on a social media corpus and using cosine similarity to find the closest neighbors of a given keyword reveals new “troll” terms which crop up over time.</li><li>k-means: running k-means clustering on embeddings for social media data reveals communities of related terms.</li></ul><p><strong>2. Forecasting: </strong>We found that twitter frequency data is often amenable to time series forecasting with models like <a href="https://www.machinelearningplus.com/time-series/arima-model-time-series-forecasting-python/#:~:text=ARIMA%2C%20short%20for%20&#39;Auto%20Regressive,used%20to%20forecast%20future%20values.">ARIMA</a>, <a href="https://online.stat.psu.edu/stat510/lesson/11/11.2">VAR</a>, and <a href="https://towardsdatascience.com/illustrated-guide-to-recurrent-neural-networks-79e5eb8049c9">RNN</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/359/1*H-zwS4tdS6_ASOf4_iz2cw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*L_HXfHj6-yJ9tMx6IOeSCw.png" /><figcaption><strong>Left: </strong>frequencies of the hashtags #fakecases, #feminism, #toxicfeminism, and #metoo during the 2017 #MeToo movement. <strong>Right: </strong>true counts and predicted counts on a daily scale for #toxic feminism. Predicted counts obtained using <strong>ARIMA(1,1,1)</strong> trained up to the (t-1)th observation. (MSE=0.688)</figcaption></figure><p>Forecasts for a given keyword can help us determine whether we should continue to use the word as a query parameter for data collection. To keep our data collection results relevant, we’d like to retain trending keywords and discard keywords which are declining in popularity.</p><p><strong>3. Meta-Data: </strong>We take into account details about a given keyword, like time since creation, sentiment, and relevance to the movement of interest. In our final algorithm, we present these details to the researcher to help them make a more informed decision about keyword selection for social media data collection.</p><p>Combining these techniques, we created a dynamic social media data streaming algorithm to track fast-changing topics and expose online trolling. I presented a poster with our ideas at the <a href="https://anqiliu-ai.github.io/files/social_projects/NeurIPS2019TwitterPosterOfficial.pdf">NeurIPS 2019 <em>AI for Social Good Workshop</em></a><em>. </em>See our <a href="https://arxiv.org/abs/1911.05332">short paper</a> for more details!</p><p>At a high level, our algorithm is as follows:</p><h4><strong>Dynamic Keyword Algorithm Overview</strong></h4><pre>Begin with initial keyword set s_0 at t=0. Repeat until t=T:</pre><pre><strong>1.</strong> Use keyword set s_t to stream dataset K_t.</pre><pre><strong>2.</strong> Train GloVe embeddings on K_t to produce G_t, the embedding space of all social media dialogue. </pre><pre><strong>3. </strong>For each word s in s_t, find n closest neighbors via cosine distance metric. </pre><pre><strong>4. </strong>Extract relevant terms from neighbors and <strong>filter</strong> outdated terms to produce s_t+1. </pre><p>The <strong>filter </strong>step in our algorithm is customizable: as of now, we factor in cosine similarity, clustering, forecasted keyword counts using ARIMA, and other metadata about the keyword (sentiment, time since creation, etc).</p><h4>Algorithm Simulation</h4><p>We simulated our algorithm on millions of historical tweets from the #MeToo movement, and I presented a poster with our findings at <a href="https://polmeth2020.org/finding-social-media-trolls-dynamic-keyword-selection-methods-rapidly-evolving-online">PolMeth 2020</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*s4WBM8C8qWuizbxEurlopQ.png" /><figcaption>dynamic keyword algorithm produced these keywords over time in our simulation run.</figcaption></figure><p>The timeline above shows the changes made to a constant-size keyword set while running the dynamic keyword algorithm on our historical #MeToo data. In practice, the keyword set size can vary — we keep it constant here to enable comparison to competitive baselines, as shown below.</p><h4>Performance</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*_4oLtIx9Z5fTDC3SIjagJg.png" /><figcaption>F1 scores for simulating dynamic keyword algorithm and other baselines on millions of historical #MeToo tweets. Each monitor used a constant-size set of 15 keywords for data collection.</figcaption></figure><p>In the figure on the left, we compare our “dynamic” monitor based on embeddings with a frequency-based “last-top” monitor, as well as a “static” monitor which retains the same keywords throughout the simulation. F1 score was calculated by comparing the 20 most popular topics in data pulled by a given monitor to the 20 most popular topics in the overall dataset. Here, we see that an embeddings-based dynamic approach can <strong>pick up </strong>topics in a fast-paced social media movement.</p><p>By utilizing cosine similarity on the embedding space, our dynamic algorithm also revealed new trolling keywords throughout the simulation, like the anti-feminist hashtags pictured below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*b96RSuKuHRYu5_kcBwHqYg.png" /><figcaption>troll terms discovered over time using the dynamic keyword algorithm on #MeToo data</figcaption></figure><h4>Next Steps</h4><p>As a next step, we are fine-tuning the parameters in our algorithm and refining our time series forecasting models. For instance, we are using models like VAR to leverage correlations between related hashtags like ‘#yesallwomen’ and ‘#notallmen’. Thanks for reading, and stay tuned for more work!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6754c8aefef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trustworthy-social-media/finding-social-media-trolls-with-dynamic-keyword-selection-b6754c8aefef">Finding Social Media Trolls with Dynamic Keyword Selection</a> was originally published in <a href="https://medium.com/trustworthy-social-media">Trustworthy Social Media</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>